-- Logs begin at Thu 2022-06-16 16:40:38 UTC, end at Thu 2022-06-16 17:50:10 UTC. -- Jun 16 17:19:49.253621 dv-cinder-837546-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 16 17:19:49.263221 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 16 17:19:49.266258 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 16 17:19:49 2022] *** Jun 16 17:19:49.266258 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 16 17:19:49.266258 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 16 17:19:49.266258 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: nodename: dv-cinder-837546-2-iscsi Jun 16 17:19:49.266258 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: machine: x86_64 Jun 16 17:19:49.266258 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: clock source: unix Jun 16 17:19:49.266258 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: pcre jit disabled Jun 16 17:19:49.266258 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: detected number of CPU cores: 4 Jun 16 17:19:49.266258 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: current working directory: / Jun 16 17:19:49.266258 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: detected binary path: /usr/bin/uwsgi-core Jun 16 17:19:49.266807 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: your processes number limit is 47680 Jun 16 17:19:49.266807 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: your memory page size is 4096 bytes Jun 16 17:19:49.266807 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: detected max file descriptor number: 2048 Jun 16 17:19:49.266807 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: lock engine: pthread robust mutexes Jun 16 17:19:49.266807 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: thunder lock: enabled Jun 16 17:19:49.266807 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 16 17:19:49.266807 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 16 17:19:49.310546 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: Python main interpreter initialized at 0x55bb832ef9c0 Jun 16 17:19:49.310546 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: python threads support enabled Jun 16 17:19:49.310546 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: your server socket listen backlog is limited to 100 connections Jun 16 17:19:49.310546 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: your mercy for graceful operations on workers is 90 seconds Jun 16 17:19:49.310927 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: mapped 403077 bytes (393 KB) for 2 cores Jun 16 17:19:49.310999 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: *** Operational MODE: preforking *** Jun 16 17:19:49.311244 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: *** uWSGI is running in multiple interpreter mode *** Jun 16 17:19:49.311244 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: spawned uWSGI master process (pid: 80142) Jun 16 17:19:49.311453 dv-cinder-837546-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 16 17:19:49.312028 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: spawned uWSGI worker 1 (pid: 80151, cores: 1) Jun 16 17:19:49.312361 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: spawned uWSGI worker 2 (pid: 80152, cores: 1) Jun 16 17:19:49.316155 dv-cinder-837546-2-iscsi devstack@keystone.service[80142]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 16 17:19:54.857954 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 17:19:54.857954 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: from cryptography.utils import int_from_bytes Jun 16 17:19:54.858774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 17:19:54.858774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: from cryptography.utils import int_from_bytes Jun 16 17:19:54.897688 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 17:19:54.897688 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: from cryptography.utils import int_from_bytes Jun 16 17:19:54.898242 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 17:19:54.898242 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: from cryptography.utils import int_from_bytes Jun 16 17:19:54.909655 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 17:19:54.909857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 17:19:54.910059 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] command line args: [] {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 17:19:54.910346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 17:19:54.910531 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ================================================================================ {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 17:19:54.910816 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] admin_token = **** {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.911071 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] config_dir = [] {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.911261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.911445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] config_source = [] {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.911630 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] debug = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.911830 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.912195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.912376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.912629 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.912880 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.913112 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] list_limit = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.913316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] log_config_append = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.913496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.913677 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] log_dir = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.913856 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] log_file = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.914097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.914351 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.914601 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.914849 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.915030 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.915209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.915456 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.916995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.917286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.917541 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.917792 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.918044 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.918306 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.919290 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.919591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.920097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.920287 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] publish_errors = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.920526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.920782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.920952 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.921228 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.921471 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.923794 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] standard_threads = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.924075 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.924871 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.925102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.925292 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] use_journal = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.925478 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] use_json = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.925661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] use_stderr = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.925845 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] use_syslog = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.926094 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.926367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.926614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.926886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.928856 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.929134 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.929418 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.929681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.929930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.930189 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.930446 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.930709 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.930954 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.931211 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.931454 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.931709 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.931962 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.932207 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.932466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.932725 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.932981 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.933246 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.933506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.933772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.934028 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.934281 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.934517 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.934751 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.934985 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.935226 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.935482 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.935734 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.935972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.936208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.936459 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.936710 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.936954 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.937228 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.937469 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.937714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.937935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.938169 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.938403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.938636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.938869 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.939108 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.939359 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.939591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.939830 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.940662 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.940892 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.941218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.941370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.941604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.941844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.942083 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.942321 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.942560 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.942802 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.943035 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.943286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.943524 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.943763 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.943996 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.944260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.944499 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.944733 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.944972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.945231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.945502 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.945742 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.945994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.946243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.946484 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.946722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.946962 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.947212 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.947453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.947693 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.947940 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.948180 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.948421 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.948615 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 17:19:54.948701 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.948821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 17:19:54.948908 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.949026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] command line args: [] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 17:19:54.949106 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.949350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 17:19:54.949433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.949536 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ================================================================================ {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 17:19:54.949619 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.949801 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] admin_token = **** {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.949887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.950056 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] config_dir = [] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.950144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.950249 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.950335 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.950435 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] config_source = [] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.950542 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.950632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] debug = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.950785 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.950865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.951024 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.951106 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.951221 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.951307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.951477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.951557 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.951709 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.951795 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.951948 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.952030 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] list_limit = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.952128 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] log_config_append = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.952210 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.952312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.952400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.952480 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] log_dir = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.952638 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.952768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] log_file = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.952893 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.953028 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.953247 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.953355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.953432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.953506 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.953659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.953740 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.953897 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.953975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.954048 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.954128 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] auth.external = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.954245 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.954348 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.954496 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.954649 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.954730 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.954884 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.954965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.955118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] auth.password = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.955202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.955350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] auth.token = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.955437 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.955622 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.955707 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.955933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.956343 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.956424 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.956588 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.956675 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.956841 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.957094 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.957194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.957356 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.957467 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] publish_errors = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.957598 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.957704 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] credential.caching = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.957840 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.957946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.958043 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.958171 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.958294 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.958395 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.958492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.958588 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.958733 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] standard_threads = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.958814 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.958974 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.959065 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.959224 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.959346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.959424 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.959543 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.959629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] use_journal = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.959734 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.959859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] use_json = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.959946 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] use_stderr = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.960019 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.960146 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] use_syslog = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.960265 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.960360 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:19:54.960516 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.960599 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.960782 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.960866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.961027 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.961116 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.961309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.961392 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.961549 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] federation.caching = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.961633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.961790 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.961883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.962039 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.962115 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.962282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.962367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.962513 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.962602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.962765 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.962846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.963012 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.963096 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.963252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.963342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.963493 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.963574 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.963739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.963818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.963975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.964064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.964225 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.964313 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.964463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] identity.caching = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.964548 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.964710 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.964800 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.964958 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.965039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.965217 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.965315 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.965457 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.965547 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.965705 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.965788 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.965937 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.966025 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.966176 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.966266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.966426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.966507 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.966678 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.966766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.966911 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.967073 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.967161 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.967237 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.967363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.967445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.967610 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.967691 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.967857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.967940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.968101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.968181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.968344 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.968466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.968588 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.968671 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.968843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.968932 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.969085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.969193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.969346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.969429 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.969580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.969660 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.969819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.969900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.970051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.970132 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.970286 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.970366 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.970525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.970604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.970762 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.970852 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.970998 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.971086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.971265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.971350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.971613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.971915 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.972013 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.972096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.972230 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.972338 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.972479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.972585 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.972745 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.972907 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.973043 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.973207 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.973337 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.973467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.973603 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.973684 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.973760 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.973834 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.973908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.973981 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.974147 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.974231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.974389 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.974471 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.974631 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.974713 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.974904 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.975005 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.975111 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.975193 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.975365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.975455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.975607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.975710 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.975849 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.975962 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.976086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.976202 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.976352 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.976434 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.976594 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.976675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.976829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.976910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.user = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.977072 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.977199 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.977339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.977421 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.977608 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.977687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.977852 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.977939 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.978096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.978176 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.978344 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.978425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.978585 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.978731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.978813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.979057 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.979183 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.979305 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.979399 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.979546 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.979627 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.979787 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.979874 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.980021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.980108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.980265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.980345 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.980507 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.980587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.980749 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.980922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.981001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.981074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.981241 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.981337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.981504 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.981590 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.981748 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.981827 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.981986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.982066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.982225 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.982307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.982463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.982543 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.982700 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.982782 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.982937 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.983017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.983179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.983259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.983414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.983495 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.983650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.983731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.983884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.983965 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.984124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.984204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.984359 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.984440 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.984597 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.984677 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.984834 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.984913 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.985079 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.985183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.985346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.985426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.985580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.985659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.985821 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.985903 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.986063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] resource.caching = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.986145 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.986315 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.986395 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] auth.external = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.986550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.986630 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.986791 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.987398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.987563 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.987819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.988058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.988144 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.988305 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.988390 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] auth.password = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.988543 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.988624 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] auth.token = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.988784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.988866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.989020 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] role.caching = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.989100 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.989289 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] role.driver = sql {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.989373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.989522 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.989605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.989806 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.989888 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.990060 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.990149 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.990310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.990392 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.990555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] credential.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.990639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.990801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.990882 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.991048 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.991130 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.991295 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.991375 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.991537 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.991616 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.991768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.991849 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.992014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.992095 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.992260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.992341 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.992506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.992594 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.992753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.992839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.993007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.993198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.993322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.993439 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.993577 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.993689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.993846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.993939 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.994098 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.994179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.994359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.994446 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.994604 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] federation.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.994691 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.994849 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.994929 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.995103 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.995184 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.995342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.995428 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.995577 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.995657 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.995811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.995897 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.996045 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.996140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.996296 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.996377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.996535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.996613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.996793 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.996880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.997034 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.997113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.997294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.997374 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.997528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.997622 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.997769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] token.caching = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.997850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.998007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.998095 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.998251 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.998339 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.998488 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.998568 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.998737 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.998815 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.998973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.999061 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.999213 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.999293 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.999461 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.999541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.999707 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.999796 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:54.999941 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.000020 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.000177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.000258 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.000413 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.000501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.000649 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.000728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.000898 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.000983 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.001234 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.001330 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.001407 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.001479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.001635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.001716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.001926 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.002006 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.002169 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.002250 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] database.connection = **** {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.002414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.002531 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.003317 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.003483 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.003776 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.004025 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.004097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.004276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.004420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.004647 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.004787 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.004915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.005039 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.005184 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.005310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.005437 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.005576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.005713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.005839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.005967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.006094 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.006220 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.006354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.006480 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.006607 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.006734 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.006858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.006984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.007117 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.007243 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.007368 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.007507 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.007633 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.007758 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.007891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.007891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80152) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 17:19:55.008129 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.008129 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.008129 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.008476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.008476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.008720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.008913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.009435 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.009672 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.009926 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.010180 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.010421 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.010663 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.010908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.011141 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.011386 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.011651 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.011888 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.012139 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.012390 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.012642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.012884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.013118 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.013387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.013630 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.013875 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.014114 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.014369 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.014621 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.014874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.015126 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.015378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.015616 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.015872 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.016115 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.016359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.016600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.016842 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.017094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.017390 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.017633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.017894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.018170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.018654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.019289 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.019538 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.019784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.020039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.020290 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.020527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.020771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.021018 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] resource.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.021304 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.021562 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.021803 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.022060 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.022318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.022561 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.022811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.023054 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.023301 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.023542 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] role.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.023792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] role.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.024029 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.024282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.024535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.024786 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.025039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.025322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.025576 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.025826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.026094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.026311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.026564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.026815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.027066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.027316 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.027772 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.028021 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.028281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.028540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.028797 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.029057 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.029327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.029593 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.029853 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.030101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.030340 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.030578 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.030816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.031075 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.031335 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.031589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.031834 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.032078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.032314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.032553 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] token.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.032796 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.033048 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.033317 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.033579 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.034040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.034757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.035023 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.035281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.035540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.035788 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.036035 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.036280 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.036535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.036791 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.037029 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.037307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.037628 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.037900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.connection = **** {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.038160 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.038414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.038659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.038908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.039152 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.039395 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.039637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.039881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.040133 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.040377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.040626 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.040870 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.041126 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.041395 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.041648 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.041884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.042133 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.042375 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:19:55.042551 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 17:19:55.124565 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.124748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.124916 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.135016 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.157202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.157441 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.157668 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.163715 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.163946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.167160 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.185983 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.186220 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.217652 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.227008 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.268979 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.269336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.272679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.272915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.341091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.341306 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.341396 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.341535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.341625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.341762 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.341846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.341982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.342066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.342199 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.342285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.342418 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.366355 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.366597 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.366824 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.367045 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.367135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.367264 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.367354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.367480 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.367569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.367696 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.367782 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.367932 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.368156 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.368379 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.378810 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.379695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.391837 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.392099 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.392281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.392370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.392507 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.392592 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.392726 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.392821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.392942 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.393028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.393223 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80152) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.393314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.393968 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.394198 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80151) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:19:55.411061 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.411272 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 17:19:55.411610 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.411811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 17:19:55.412085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.412297 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.412384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.412513 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 17:19:55.412596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.412837 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.412928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.413045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 17:19:55.413346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.413614 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.413877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.414141 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.423662 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.423936 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.424205 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.425051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.426204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.426489 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.426766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.427052 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.433332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80152) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.435392 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.436620 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80152) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.436899 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.437210 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.437496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.437770 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.438031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.438298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.438570 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.438733 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.438838 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.439017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.439101 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.439304 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.439385 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.439589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.439871 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.440666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.440956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.441264 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.441543 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.441820 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.442090 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.465948 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.467555 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80152) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.467856 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.469434 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.471067 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.472081 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.473903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80152) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.474299 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.474580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.477795 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.478081 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.478476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.481898 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80152) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.482179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.485887 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.486176 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.486665 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80152) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.487054 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.487316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.487679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.487960 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.488235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.488505 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.488776 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.491274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.491571 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.491854 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.492138 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.492417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.492697 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.492971 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.493268 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.508284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.508559 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.513421 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80152) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.515902 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.516191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.517078 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.517499 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.521900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.523109 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80152) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.525607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.525908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.526736 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80152) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.530432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.530729 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.531013 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.531135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.531295 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.531575 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.531857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.535418 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.539198 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.539502 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.539791 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.540080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.540361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.540641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.551781 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80152) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.552064 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.552336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.552597 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.560525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.560977 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.561289 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.561564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.561832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.562278 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.562561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.567623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80152) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.567902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.568182 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.568469 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.568751 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.570895 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.572641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.572930 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.577472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.577760 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.578051 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.578341 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.578636 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.582248 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80152) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.582530 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.582817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.583094 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.583469 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.583751 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.584030 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.592651 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.592937 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.593247 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.593536 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.593942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.594198 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.594482 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.602134 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80152) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.605421 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80152) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.608723 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.610495 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.612672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80152) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.612949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.613256 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.614144 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.617487 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.620571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.620846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.622242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80152) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.622666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.624347 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.624636 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.624928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.625589 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.625883 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.626288 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.626567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.634705 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.636284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.636801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80152) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.637092 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.637414 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.639597 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.639896 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.640299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.640584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.645487 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80152) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.645769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.646049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.646325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.646595 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.646991 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.647285 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.647654 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.647936 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.648206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.648477 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.648739 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.649013 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.649306 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80152) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.650539 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.650845 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.651132 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.659671 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80151) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:19:55.659953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.660240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.660524 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.660803 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.661230 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.661519 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.661801 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.662085 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.662360 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.662633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.662912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.663183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.663446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80151) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:19:55.687595 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55bb832ef9c0 pid: 80152 (default app) Jun 16 17:19:55.691805 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a33425-9f48-455f-b79b-69cb3733a3e2 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:19:55.692099 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a33425-9f48-455f-b79b-69cb3733a3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:19:55.692383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a33425-9f48-455f-b79b-69cb3733a3e2 None None] PATH_INFO: `/v3` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:19:55.696788 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1/1] 10.222.0.40 () {50 vars in 791 bytes} [Thu Jun 16 17:19:55 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:19:55.703682 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55bb832ef9c0 pid: 80151 (default app) Jun 16 17:20:03.559730 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72012239-3962-4c7b-bbb3-fb411f222c2a None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:03.560291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72012239-3962-4c7b-bbb3-fb411f222c2a None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:03.560291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72012239-3962-4c7b-bbb3-fb411f222c2a None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:03.561715 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 2/2] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:20:03.570426 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec565dd-b609-41b5-8d93-c9617bb851d9 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:03.570912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec565dd-b609-41b5-8d93-c9617bb851d9 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:03.571402 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec565dd-b609-41b5-8d93-c9617bb851d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:03.643453 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: INFO dbcounter [None req-aec565dd-b609-41b5-8d93-c9617bb851d9 None None] Registered counter for database keystone Jun 16 17:20:03.664904 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG oslo_db.sqlalchemy.engines [None req-aec565dd-b609-41b5-8d93-c9617bb851d9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80151) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 17:20:03.671133 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writer thread running {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 17:20:03.857022 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG passlib.handlers.bcrypt [None req-aec565dd-b609-41b5-8d93-c9617bb851d9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80151) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 17:20:03.857401 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG passlib.handlers.bcrypt [None req-aec565dd-b609-41b5-8d93-c9617bb851d9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80151) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 17:20:03.912854 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-aec565dd-b609-41b5-8d93-c9617bb851d9 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:03.914534 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-aec565dd-b609-41b5-8d93-c9617bb851d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:04.038382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1/3] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 17:20:04.223680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72012239-3962-4c7b-bbb3-fb411f222c2a None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:04.295718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: INFO dbcounter [None req-72012239-3962-4c7b-bbb3-fb411f222c2a None None] Registered counter for database keystone Jun 16 17:20:04.316029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG oslo_db.sqlalchemy.engines [None req-72012239-3962-4c7b-bbb3-fb411f222c2a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80152) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 17:20:04.322325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writer thread running {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 17:20:04.446117 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72012239-3962-4c7b-bbb3-fb411f222c2a None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:04.450370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60798b-ff77-4624-a616-6a4a86489aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:04.451041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60798b-ff77-4624-a616-6a4a86489aca admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:04.451336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60798b-ff77-4624-a616-6a4a86489aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:04.451620 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60798b-ff77-4624-a616-6a4a86489aca admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:04.460331 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60798b-ff77-4624-a616-6a4a86489aca admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:04.512059 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1a60798b-ff77-4624-a616-6a4a86489aca admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 17:20:04.512341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60798b-ff77-4624-a616-6a4a86489aca admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:04.522721 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 3/4] 10.222.0.40 () {58 vars in 1324 bytes} [Thu Jun 16 17:20:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:20:04.528631 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec565dd-b609-41b5-8d93-c9617bb851d9 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:04.545426 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec565dd-b609-41b5-8d93-c9617bb851d9 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:04.550527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a179f-0074-4807-b4e4-b9e60411a991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:04.551641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a179f-0074-4807-b4e4-b9e60411a991 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:04.551932 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a179f-0074-4807-b4e4-b9e60411a991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:04.552211 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a179f-0074-4807-b4e4-b9e60411a991 admin admin] PATH_INFO: `/v3/projects/26ada52fc9b44bcbbe7f2cc5f37ab303` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:04.553755 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a179f-0074-4807-b4e4-b9e60411a991 admin admin] RBAC: Authorizing `identity:get_project(project_id=26ada52fc9b44bcbbe7f2cc5f37ab303)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:04.603627 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ef6a179f-0074-4807-b4e4-b9e60411a991 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 17:20:04.604193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a179f-0074-4807-b4e4-b9e60411a991 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:04.608603 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 2/5] 10.222.0.40 () {56 vars in 1207 bytes} [Thu Jun 16 17:20:04 2022] GET /identity/v3/projects/26ada52fc9b44bcbbe7f2cc5f37ab303 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:06.701676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2da743-5c30-4bbd-a13f-e8b9152d1b92 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:06.702284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2da743-5c30-4bbd-a13f-e8b9152d1b92 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:06.702284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2da743-5c30-4bbd-a13f-e8b9152d1b92 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:06.706407 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 4/6] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:06.712275 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcba4d-899c-45cc-8344-bcab0668e4ea None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:06.712716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcba4d-899c-45cc-8344-bcab0668e4ea None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:06.712838 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcba4d-899c-45cc-8344-bcab0668e4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:06.731964 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-30fcba4d-899c-45cc-8344-bcab0668e4ea None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:06.733556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-30fcba4d-899c-45cc-8344-bcab0668e4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:06.742346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 3/7] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 16 17:20:06.765166 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2da743-5c30-4bbd-a13f-e8b9152d1b92 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:06.778332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2da743-5c30-4bbd-a13f-e8b9152d1b92 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:06.782528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795b4a2-0e18-405e-b369-bdd52ea3f635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:06.783169 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795b4a2-0e18-405e-b369-bdd52ea3f635 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:06.783459 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795b4a2-0e18-405e-b369-bdd52ea3f635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:06.783749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795b4a2-0e18-405e-b369-bdd52ea3f635 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:06.791869 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795b4a2-0e18-405e-b369-bdd52ea3f635 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:06.793685 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795b4a2-0e18-405e-b369-bdd52ea3f635 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:06.802279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 5/8] 10.222.0.40 () {58 vars in 1324 bytes} [Thu Jun 16 17:20:06 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:20:06.807622 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fcba4d-899c-45cc-8344-bcab0668e4ea None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:06.816926 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fcba4d-899c-45cc-8344-bcab0668e4ea None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:06.821417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f5668-58e4-4cf7-8827-311e09cbd5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:06.822292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f5668-58e4-4cf7-8827-311e09cbd5cd admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:06.822586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f5668-58e4-4cf7-8827-311e09cbd5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:06.822871 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f5668-58e4-4cf7-8827-311e09cbd5cd admin admin] PATH_INFO: `/v3/users/eac0995e068547e3890aa6066bdecd84` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:06.824277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f5668-58e4-4cf7-8827-311e09cbd5cd admin admin] RBAC: Authorizing `identity:get_user(user_id=eac0995e068547e3890aa6066bdecd84)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:06.826166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f5668-58e4-4cf7-8827-311e09cbd5cd admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:06.828196 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 4/9] 10.222.0.40 () {56 vars in 1198 bytes} [Thu Jun 16 17:20:06 2022] GET /identity/v3/users/eac0995e068547e3890aa6066bdecd84 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:09.046522 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe31dd3-a0fd-4941-a035-23321afce7a8 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:09.047071 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe31dd3-a0fd-4941-a035-23321afce7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:09.047071 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe31dd3-a0fd-4941-a035-23321afce7a8 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:09.048637 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 6/10] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:09.051062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043d2c8-395b-4c65-bbc2-7c619bd8a387 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:09.051477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043d2c8-395b-4c65-bbc2-7c619bd8a387 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:09.051627 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043d2c8-395b-4c65-bbc2-7c619bd8a387 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:09.052985 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 5/11] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:09.054836 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f0262-fac6-4a11-a230-ca0a1ffce721 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:09.055140 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f0262-fac6-4a11-a230-ca0a1ffce721 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:09.055431 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f0262-fac6-4a11-a230-ca0a1ffce721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:09.057927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52a140-4bb3-48b5-82a7-08fef2ad3a6c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:09.058222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52a140-4bb3-48b5-82a7-08fef2ad3a6c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:09.058507 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52a140-4bb3-48b5-82a7-08fef2ad3a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:09.075380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-ee52a140-4bb3-48b5-82a7-08fef2ad3a6c None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:09.076564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ee52a140-4bb3-48b5-82a7-08fef2ad3a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:09.096144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG passlib.handlers.bcrypt [None req-c09f0262-fac6-4a11-a230-ca0a1ffce721 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80152) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 17:20:09.097172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG passlib.handlers.bcrypt [None req-c09f0262-fac6-4a11-a230-ca0a1ffce721 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80152) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 17:20:09.144673 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-c09f0262-fac6-4a11-a230-ca0a1ffce721 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:09.146319 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-c09f0262-fac6-4a11-a230-ca0a1ffce721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:09.156806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 6/12] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:20:09.180369 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52a140-4bb3-48b5-82a7-08fef2ad3a6c None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:09.204882 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:09.247827 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 7/13] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:09.269949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f0262-fac6-4a11-a230-ca0a1ffce721 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:09.291744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52a140-4bb3-48b5-82a7-08fef2ad3a6c None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:09.327246 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e63b53-ab32-4e98-8b6d-c834696b3cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:09.327981 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e63b53-ab32-4e98-8b6d-c834696b3cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:09.328269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e63b53-ab32-4e98-8b6d-c834696b3cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:09.328556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e63b53-ab32-4e98-8b6d-c834696b3cf5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:09.340119 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e63b53-ab32-4e98-8b6d-c834696b3cf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:09.351790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-49e63b53-ab32-4e98-8b6d-c834696b3cf5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:20:09.351790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:09.351790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:09.351790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:09.351790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:09.351790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:09.351790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:09.351790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:09.351790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:09.351790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:09.351790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:09.351790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:09.351790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:09.351790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:09.351790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:09.351790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:09.352774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:09.352774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:09.352774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:09.352774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:09.352774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:09.352774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:09.352774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:09.352774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:09.352774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:09.352774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:09.352774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:09.352774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:09.352774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:09.352774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:09.352774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:09.352774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:09.353673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:09.353673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:09.353673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:09.353673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:09.353673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:09.353673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:09.353673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:09.353673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:09.353673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:09.353673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:09.353673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:20:09.353673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:20:09.353673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 7/14] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:20:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:20:09.358719 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e63b53-ab32-4e98-8b6d-c834696b3cf5 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:09.371827 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f0262-fac6-4a11-a230-ca0a1ffce721 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:09.374278 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:09.412402 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca7200-7792-46a3-90da-2adcc4883bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:09.413193 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca7200-7792-46a3-90da-2adcc4883bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:09.413625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca7200-7792-46a3-90da-2adcc4883bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:09.413625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca7200-7792-46a3-90da-2adcc4883bd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:09.443404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:09.462472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e63b53-ab32-4e98-8b6d-c834696b3cf5 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:09.483282 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca7200-7792-46a3-90da-2adcc4883bd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:09.484834 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca7200-7792-46a3-90da-2adcc4883bd9 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:09.496716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472b4e9-1a2a-40f9-8a9b-6654b7476b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:09.497420 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472b4e9-1a2a-40f9-8a9b-6654b7476b2b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:09.497712 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472b4e9-1a2a-40f9-8a9b-6654b7476b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:09.497996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472b4e9-1a2a-40f9-8a9b-6654b7476b2b None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:09.498757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2472b4e9-1a2a-40f9-8a9b-6654b7476b2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:09.499073 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2472b4e9-1a2a-40f9-8a9b-6654b7476b2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:09.500472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2472b4e9-1a2a-40f9-8a9b-6654b7476b2b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:09.513127 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 8/15] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:09.519294 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472b4e9-1a2a-40f9-8a9b-6654b7476b2b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:09.584097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 8/16] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:20:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:20:09.589660 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca7200-7792-46a3-90da-2adcc4883bd9 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:09.592351 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:09.620465 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472b4e9-1a2a-40f9-8a9b-6654b7476b2b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:09.662574 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98d3a6-4439-4823-9c12-e8b98be67339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:09.663457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98d3a6-4439-4823-9c12-e8b98be67339 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:09.663748 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98d3a6-4439-4823-9c12-e8b98be67339 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:09.664032 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98d3a6-4439-4823-9c12-e8b98be67339 None admin] PATH_INFO: `/v3/users/eac0995e068547e3890aa6066bdecd84` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:09.665293 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98d3a6-4439-4823-9c12-e8b98be67339 None admin] RBAC: Authorizing `identity:get_user(user_id=eac0995e068547e3890aa6066bdecd84)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:09.666770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98d3a6-4439-4823-9c12-e8b98be67339 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:09.668411 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 9/17] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:09 2022] GET /identity/v3/users/eac0995e068547e3890aa6066bdecd84 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:09.674119 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98d3a6-4439-4823-9c12-e8b98be67339 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:09.688321 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:09.693231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca7200-7792-46a3-90da-2adcc4883bd9 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:09.728951 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f1492-9c74-41a2-8636-bce8dc939105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:09.729875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f1492-9c74-41a2-8636-bce8dc939105 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:09.730166 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f1492-9c74-41a2-8636-bce8dc939105 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:09.730449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f1492-9c74-41a2-8636-bce8dc939105 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:09.748646 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f1492-9c74-41a2-8636-bce8dc939105 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:09.750174 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f1492-9c74-41a2-8636-bce8dc939105 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:09.772342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-425f1492-9c74-41a2-8636-bce8dc939105 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:20:09.772342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:09.772342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:09.772342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:09.772342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:09.772342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:09.772342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:09.772342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:09.772342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:09.772342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:09.772342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:09.772342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:09.772342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:20:09.772342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:20:09.772342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:20:09.773403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:20:09.773403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:09.773403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:09.773403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:09.773403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:09.773403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:09.773403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:09.773403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:09.773403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:09.773403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:09.773403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:09.773403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:09.773403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:09.773403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:09.773403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:09.773403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:09.773403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:09.774419 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:20:09.774419 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:20:09.774419 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:20:09.774419 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:20:09.774419 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 9/18] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:20:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:20:09.777599 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98d3a6-4439-4823-9c12-e8b98be67339 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:09.779313 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f1492-9c74-41a2-8636-bce8dc939105 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:09.812147 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ec9b8-3554-4888-80d7-539312630c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:09.812954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ec9b8-3554-4888-80d7-539312630c19 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:09.813272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ec9b8-3554-4888-80d7-539312630c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:09.813553 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ec9b8-3554-4888-80d7-539312630c19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:09.815005 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ec9b8-3554-4888-80d7-539312630c19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:09.817850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ec9b8-3554-4888-80d7-539312630c19 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:09.819488 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 10/19] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:20:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:09.825083 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ec9b8-3554-4888-80d7-539312630c19 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:09.837653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:09.882817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f1492-9c74-41a2-8636-bce8dc939105 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:09.905806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:09.916348 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9fc22-a7c7-4411-b1ef-7453b75afff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:09.917018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9fc22-a7c7-4411-b1ef-7453b75afff5 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:09.917343 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9fc22-a7c7-4411-b1ef-7453b75afff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:09.917626 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9fc22-a7c7-4411-b1ef-7453b75afff5 None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:09.918355 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9fc22-a7c7-4411-b1ef-7453b75afff5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:09.918668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9fc22-a7c7-4411-b1ef-7453b75afff5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:09.920046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9fc22-a7c7-4411-b1ef-7453b75afff5 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:09.925738 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ec9b8-3554-4888-80d7-539312630c19 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:09.937644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 10/20] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:09.961014 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfca550-25fe-475d-86a9-ad39bdc60130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:09.961738 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfca550-25fe-475d-86a9-ad39bdc60130 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:09.962025 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfca550-25fe-475d-86a9-ad39bdc60130 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:09.962306 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfca550-25fe-475d-86a9-ad39bdc60130 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:09.963168 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfca550-25fe-475d-86a9-ad39bdc60130 None admin] RBAC: Adding query filter params (role.id=25a16d8ea4084e9eacdebb7a4dcd2337, scope.domain.id=default, user.id=eac0995e068547e3890aa6066bdecd84) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:09.963485 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfca550-25fe-475d-86a9-ad39bdc60130 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:09.964985 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfca550-25fe-475d-86a9-ad39bdc60130 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:09.972846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 11/21] 10.222.0.40 () {56 vars in 1322 bytes} [Thu Jun 16 17:20:09 2022] GET /identity/v3/role_assignments?user.id=eac0995e068547e3890aa6066bdecd84&scope.domain.id=default&role.id=25a16d8ea4084e9eacdebb7a4dcd2337 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:12.656423 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd74e60-8197-4194-8067-8874571ee933 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:12.656956 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd74e60-8197-4194-8067-8874571ee933 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:12.657118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd74e60-8197-4194-8067-8874571ee933 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:12.658552 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 11/22] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:12.663738 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40faff7-6e5a-42fe-b460-7f00cc299928 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:12.664159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40faff7-6e5a-42fe-b460-7f00cc299928 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:12.665665 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40faff7-6e5a-42fe-b460-7f00cc299928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:12.698187 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a40faff7-6e5a-42fe-b460-7f00cc299928 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:12.699704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a40faff7-6e5a-42fe-b460-7f00cc299928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:12.844221 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc03cd-a0fc-4ebb-b5cc-e2471b5b7e85 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:12.844533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc03cd-a0fc-4ebb-b5cc-e2471b5b7e85 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:12.844800 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc03cd-a0fc-4ebb-b5cc-e2471b5b7e85 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:12.846539 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 12/23] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:12.851599 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8ba1d-ef33-4940-9f6d-1f89c71f19e5 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:12.857172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8ba1d-ef33-4940-9f6d-1f89c71f19e5 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:12.857172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8ba1d-ef33-4940-9f6d-1f89c71f19e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:12.896636 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 12/24] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:12.897803 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-11d8ba1d-ef33-4940-9f6d-1f89c71f19e5 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:12.899100 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-11d8ba1d-ef33-4940-9f6d-1f89c71f19e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:12.928416 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40faff7-6e5a-42fe-b460-7f00cc299928 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:13.145569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:13.154397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 13/25] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:20:13.155742 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3399c-5f8a-448c-b1be-5af0285ccd27 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:13.156033 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3399c-5f8a-448c-b1be-5af0285ccd27 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:13.156313 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3399c-5f8a-448c-b1be-5af0285ccd27 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:13.157745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 14/26] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:13.162506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3c0d2-c93d-479f-b264-ecbfe55e5443 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:13.163921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3c0d2-c93d-479f-b264-ecbfe55e5443 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:13.166932 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3c0d2-c93d-479f-b264-ecbfe55e5443 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:13.168383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 15/27] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:13 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:13.175694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e7421-46e9-4af0-a0d4-4dc7591960ae None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:13.176335 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e7421-46e9-4af0-a0d4-4dc7591960ae None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:13.178103 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e7421-46e9-4af0-a0d4-4dc7591960ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:13.188196 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40faff7-6e5a-42fe-b460-7f00cc299928 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:13.196339 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-e42e7421-46e9-4af0-a0d4-4dc7591960ae None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:13.197604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-e42e7421-46e9-4af0-a0d4-4dc7591960ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:13.226235 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f82a35-edaf-4e86-b117-b85656235f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:13.227412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f82a35-edaf-4e86-b117-b85656235f6d None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:13.228208 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f82a35-edaf-4e86-b117-b85656235f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:13.228955 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f82a35-edaf-4e86-b117-b85656235f6d None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:13.229952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f82a35-edaf-4e86-b117-b85656235f6d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:13.231560 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f82a35-edaf-4e86-b117-b85656235f6d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:13.262589 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:13.291182 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 16/28] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:13.293021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af8cb3-f880-46aa-9527-56c46f0e30bb None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:13.293730 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af8cb3-f880-46aa-9527-56c46f0e30bb None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:13.294225 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af8cb3-f880-46aa-9527-56c46f0e30bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:13.312067 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-18af8cb3-f880-46aa-9527-56c46f0e30bb None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:13.313637 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-18af8cb3-f880-46aa-9527-56c46f0e30bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:13.377129 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 13/29] 10.222.0.40 () {60 vars in 1131 bytes} [Thu Jun 16 17:20:12 2022] POST /identity/v3/roles => generated 219 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:20:13.378790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f82a35-edaf-4e86-b117-b85656235f6d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:13.403284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 17/30] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:13.404171 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af8cb3-f880-46aa-9527-56c46f0e30bb None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:13.562317 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af8cb3-f880-46aa-9527-56c46f0e30bb None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:13.577489 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:13.622920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed43257-2085-4a72-942d-9410ace71630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:13.623617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed43257-2085-4a72-942d-9410ace71630 None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:13.623908 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed43257-2085-4a72-942d-9410ace71630 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:13.624311 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed43257-2085-4a72-942d-9410ace71630 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:13.625085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed43257-2085-4a72-942d-9410ace71630 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:13.626546 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed43257-2085-4a72-942d-9410ace71630 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:13.691933 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f82a35-edaf-4e86-b117-b85656235f6d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:13.735236 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd245f8-0474-42ae-8d16-7fd22844ec2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:13.736532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd245f8-0474-42ae-8d16-7fd22844ec2f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:13.737065 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd245f8-0474-42ae-8d16-7fd22844ec2f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:13.737601 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd245f8-0474-42ae-8d16-7fd22844ec2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:13.741458 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:13.753314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd245f8-0474-42ae-8d16-7fd22844ec2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:13.755535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd245f8-0474-42ae-8d16-7fd22844ec2f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:13.779311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-4dd245f8-0474-42ae-8d16-7fd22844ec2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:20:13.779311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:13.779311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:13.779311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:13.779311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:13.779311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:13.779311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:13.779311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:13.779311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:13.779311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:13.779311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:13.779311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:13.779311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:20:13.779311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:20:13.779311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:20:13.780210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:20:13.780210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:13.780210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:13.780210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:13.780210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:13.780210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:13.780210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:13.780210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:13.780210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:13.780210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:13.780210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:13.780210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:13.780210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:13.780210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:13.780210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:13.780210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:13.780210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:13.781050 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:20:13.781050 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:20:13.781050 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:20:13.781050 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:20:13.783149 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 14/31] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:20:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:13.783263 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 18/32] 10.222.0.40 () {60 vars in 1131 bytes} [Thu Jun 16 17:20:13 2022] POST /identity/v3/roles => generated 225 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:20:13.784805 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed43257-2085-4a72-942d-9410ace71630 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:13.791136 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd245f8-0474-42ae-8d16-7fd22844ec2f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:13.925213 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed43257-2085-4a72-942d-9410ace71630 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:13.941104 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd245f8-0474-42ae-8d16-7fd22844ec2f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:13.966264 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17ccc5-8cdd-4490-932a-ed38d4d9f6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:13.967435 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17ccc5-8cdd-4490-932a-ed38d4d9f6e0 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:13.968079 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17ccc5-8cdd-4490-932a-ed38d4d9f6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:13.968354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17ccc5-8cdd-4490-932a-ed38d4d9f6e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:13.985733 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17ccc5-8cdd-4490-932a-ed38d4d9f6e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:13.991059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f18a3-7010-4cdc-890a-83e9675846b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:13.991565 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f18a3-7010-4cdc-890a-83e9675846b3 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:13.991864 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f18a3-7010-4cdc-890a-83e9675846b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:13.992156 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f18a3-7010-4cdc-890a-83e9675846b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:13.993218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f18a3-7010-4cdc-890a-83e9675846b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:13.993742 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f18a3-7010-4cdc-890a-83e9675846b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:13.994886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-3c17ccc5-8cdd-4490-932a-ed38d4d9f6e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:20:13.994886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:13.994886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:13.994886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:13.994886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:13.994886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:13.994886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:13.994886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:13.994886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:13.994886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:13.994886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:13.994886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:13.994886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:13.994886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:13.994886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:13.994886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:13.995722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:13.995722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:13.995722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:13.995722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:13.995722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:13.995722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:13.995722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:13.995722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:13.995722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:13.995722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:13.995722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:13.995722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:13.995722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:13.995722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:13.995722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:13.995722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:13.996533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:13.996533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:13.996533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:13.996533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:13.996533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:13.996533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:13.996533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:13.996533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:13.996533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:13.996533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:13.996533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:20:13.996533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:20:13.997164 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f18a3-7010-4cdc-890a-83e9675846b3 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:13.998079 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 19/33] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:20:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:13.998079 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7261d-774d-4256-99fc-baf1bc627c32 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:13.998079 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7261d-774d-4256-99fc-baf1bc627c32 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:13.998356 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7261d-774d-4256-99fc-baf1bc627c32 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:14.001254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 20/34] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:14.007491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7261d-774d-4256-99fc-baf1bc627c32 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:14.026724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 15/35] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:14.028201 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f98aba-fde3-47a5-bb1b-1df9a2152e79 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:14.028504 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f98aba-fde3-47a5-bb1b-1df9a2152e79 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:14.028800 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f98aba-fde3-47a5-bb1b-1df9a2152e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:14.049179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:14.055122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-f0f98aba-fde3-47a5-bb1b-1df9a2152e79 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:14.057899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f0f98aba-fde3-47a5-bb1b-1df9a2152e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:14.111366 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:14.137333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7261d-774d-4256-99fc-baf1bc627c32 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:14.145195 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 16/36] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:14.145195 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f98aba-fde3-47a5-bb1b-1df9a2152e79 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:14.179195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960386f-eac9-4e70-887c-52788622b8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:14.179798 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960386f-eac9-4e70-887c-52788622b8ed None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:14.180095 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960386f-eac9-4e70-887c-52788622b8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:14.180384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960386f-eac9-4e70-887c-52788622b8ed None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:14.181174 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960386f-eac9-4e70-887c-52788622b8ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:14.181516 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960386f-eac9-4e70-887c-52788622b8ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:14.183057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960386f-eac9-4e70-887c-52788622b8ed None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:14.196362 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 21/37] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:14.201365 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960386f-eac9-4e70-887c-52788622b8ed None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:14.259808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f98aba-fde3-47a5-bb1b-1df9a2152e79 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:14.300105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0873f-ffde-4b1d-b71e-f945df8c9eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:14.300819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0873f-ffde-4b1d-b71e-f945df8c9eb0 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:14.301108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0873f-ffde-4b1d-b71e-f945df8c9eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:14.301420 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0873f-ffde-4b1d-b71e-f945df8c9eb0 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:14.302147 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee0873f-ffde-4b1d-b71e-f945df8c9eb0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:14.303624 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee0873f-ffde-4b1d-b71e-f945df8c9eb0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:14.306871 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:14.323174 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960386f-eac9-4e70-887c-52788622b8ed None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:14.334940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 17/38] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 17:20:14 2022] POST /identity/v3/projects => generated 301 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:20:14.336620 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0873f-ffde-4b1d-b71e-f945df8c9eb0 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:14.367721 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee8166-ed32-4290-b390-1a080e21887d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:14.368531 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee8166-ed32-4290-b390-1a080e21887d None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:14.368904 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee8166-ed32-4290-b390-1a080e21887d None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:14.369297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee8166-ed32-4290-b390-1a080e21887d None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:14.370309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee8166-ed32-4290-b390-1a080e21887d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:14.372015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee8166-ed32-4290-b390-1a080e21887d None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:14.397214 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 22/39] 10.222.0.40 () {60 vars in 1131 bytes} [Thu Jun 16 17:20:14 2022] POST /identity/v3/roles => generated 223 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:20:14.402756 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:14.406323 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56a752-e9b7-4b04-846c-952096de5fe0 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:14.406617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56a752-e9b7-4b04-846c-952096de5fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:14.407387 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56a752-e9b7-4b04-846c-952096de5fe0 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:14.409769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 23/40] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:20:14.415588 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae06e8-76f1-4d6b-a1e7-b0013812a895 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:14.415917 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae06e8-76f1-4d6b-a1e7-b0013812a895 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:14.416320 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae06e8-76f1-4d6b-a1e7-b0013812a895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:14.516628 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0873f-ffde-4b1d-b71e-f945df8c9eb0 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:14.552981 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-eeae06e8-76f1-4d6b-a1e7-b0013812a895 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:14.555667 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-eeae06e8-76f1-4d6b-a1e7-b0013812a895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:14.567053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad24ec2-23c5-43e9-938b-b88b78957d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:14.568067 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad24ec2-23c5-43e9-938b-b88b78957d1d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:14.568366 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad24ec2-23c5-43e9-938b-b88b78957d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:14.568654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad24ec2-23c5-43e9-938b-b88b78957d1d None admin] PATH_INFO: `/v3/users/eac0995e068547e3890aa6066bdecd84` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:14.570283 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad24ec2-23c5-43e9-938b-b88b78957d1d None admin] RBAC: Authorizing `identity:get_user(user_id=eac0995e068547e3890aa6066bdecd84)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:14.571741 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad24ec2-23c5-43e9-938b-b88b78957d1d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:14.573372 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 18/41] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:14 2022] GET /identity/v3/users/eac0995e068547e3890aa6066bdecd84 => generated 247 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:14.578815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad24ec2-23c5-43e9-938b-b88b78957d1d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:14.645363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 24/42] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:20:14.668278 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae06e8-76f1-4d6b-a1e7-b0013812a895 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:14.693619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:14.699951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad24ec2-23c5-43e9-938b-b88b78957d1d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:14.734808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2756db-c1d6-4b00-8319-ae15c4f49d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:14.735611 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2756db-c1d6-4b00-8319-ae15c4f49d37 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:14.735902 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2756db-c1d6-4b00-8319-ae15c4f49d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:14.736187 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2756db-c1d6-4b00-8319-ae15c4f49d37 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:14.737989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2756db-c1d6-4b00-8319-ae15c4f49d37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:14.739434 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2756db-c1d6-4b00-8319-ae15c4f49d37 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:14.741065 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 19/43] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:20:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:14.746546 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2756db-c1d6-4b00-8319-ae15c4f49d37 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:14.747760 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:14.770722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae06e8-76f1-4d6b-a1e7-b0013812a895 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:14.806176 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b7272-7621-4eb4-87aa-3c823476daa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:14.806994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b7272-7621-4eb4-87aa-3c823476daa6 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:14.807286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b7272-7621-4eb4-87aa-3c823476daa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:14.807575 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b7272-7621-4eb4-87aa-3c823476daa6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:14.809014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b7272-7621-4eb4-87aa-3c823476daa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:14.810526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b7272-7621-4eb4-87aa-3c823476daa6 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:14.812176 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 25/44] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:20:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:14.817776 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b7272-7621-4eb4-87aa-3c823476daa6 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:14.853207 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2756db-c1d6-4b00-8319-ae15c4f49d37 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:14.888504 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74332d29-ba0d-4757-b43e-b678609b2cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:14.889424 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74332d29-ba0d-4757-b43e-b678609b2cd0 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:14.889724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74332d29-ba0d-4757-b43e-b678609b2cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:14.890018 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74332d29-ba0d-4757-b43e-b678609b2cd0 None admin] PATH_INFO: `/v3/domains/default/users/eac0995e068547e3890aa6066bdecd84/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:14.892082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74332d29-ba0d-4757-b43e-b678609b2cd0 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=eac0995e068547e3890aa6066bdecd84, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:14.893907 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74332d29-ba0d-4757-b43e-b678609b2cd0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:14.907516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 20/45] 10.222.0.40 () {58 vars in 1361 bytes} [Thu Jun 16 17:20:14 2022] PUT /identity/v3/domains/default/users/eac0995e068547e3890aa6066bdecd84/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:20:14.926350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b7272-7621-4eb4-87aa-3c823476daa6 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:14.962478 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac23be-09d1-4a0c-9624-f3d53414d2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:14.963310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac23be-09d1-4a0c-9624-f3d53414d2a0 None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:14.963675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac23be-09d1-4a0c-9624-f3d53414d2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:14.964026 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac23be-09d1-4a0c-9624-f3d53414d2a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:14.964844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac23be-09d1-4a0c-9624-f3d53414d2a0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:14.966528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac23be-09d1-4a0c-9624-f3d53414d2a0 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:14.990504 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:14.998866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 26/46] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 17:20:14 2022] POST /identity/v3/projects => generated 312 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:20:17.043350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99243af6-bb0e-40cb-a86b-47498130d30d None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:17.043876 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99243af6-bb0e-40cb-a86b-47498130d30d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:17.044049 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99243af6-bb0e-40cb-a86b-47498130d30d None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:17.045498 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 21/47] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:17.050955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bf257-af26-421d-9edb-c70079439e88 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:17.051382 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bf257-af26-421d-9edb-c70079439e88 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:17.051549 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bf257-af26-421d-9edb-c70079439e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:17.073704 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-236bf257-af26-421d-9edb-c70079439e88 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:17.075238 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-236bf257-af26-421d-9edb-c70079439e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:17.148666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9d735-f68c-4424-afda-51e5b40412c0 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:17.148957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9d735-f68c-4424-afda-51e5b40412c0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:17.149275 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9d735-f68c-4424-afda-51e5b40412c0 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:17.150673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 22/48] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:20:17.153390 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 27/49] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:17.155462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7d714-557a-40c0-85dc-3a6a8e160b69 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:17.155859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7d714-557a-40c0-85dc-3a6a8e160b69 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:17.156065 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7d714-557a-40c0-85dc-3a6a8e160b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:17.172900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-26f7d714-557a-40c0-85dc-3a6a8e160b69 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:17.174233 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-26f7d714-557a-40c0-85dc-3a6a8e160b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:17.177871 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236bf257-af26-421d-9edb-c70079439e88 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:17.192245 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:17.247384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 23/50] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:17.271097 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7d714-557a-40c0-85dc-3a6a8e160b69 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:17.281439 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236bf257-af26-421d-9edb-c70079439e88 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:17.308332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:17.318237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797dd0d-5368-4651-b0cb-6a58c1faaf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:17.319051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797dd0d-5368-4651-b0cb-6a58c1faaf90 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:17.319344 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797dd0d-5368-4651-b0cb-6a58c1faaf90 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:17.319623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797dd0d-5368-4651-b0cb-6a58c1faaf90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:17.330169 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797dd0d-5368-4651-b0cb-6a58c1faaf90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:17.338279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-4797dd0d-5368-4651-b0cb-6a58c1faaf90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:20:17.338279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:17.338279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:17.338279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:17.338279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:17.338279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:17.338279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:17.338279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:17.338279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:17.338279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:17.338279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:17.338279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:17.338279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:17.338279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:17.338279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:17.338279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:17.339129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:17.339129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:17.339129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:17.339129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:17.339129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:17.339129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:17.339129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:17.339129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:17.339129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:17.339129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:17.339129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:17.339129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:17.339129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:17.339129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:17.339129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:17.339129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:17.339920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:17.339920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:17.339920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:17.339920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:17.339920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:17.339920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:17.339920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:17.339920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:17.339920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:17.339920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:17.339920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:20:17.339920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:20:17.339920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 28/51] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:20:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:20:17.345258 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797dd0d-5368-4651-b0cb-6a58c1faaf90 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:17.372371 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7d714-557a-40c0-85dc-3a6a8e160b69 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:17.410705 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a138b-38da-459a-8b4c-10b3ce0d63af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:17.412820 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a138b-38da-459a-8b4c-10b3ce0d63af None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:17.413117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a138b-38da-459a-8b4c-10b3ce0d63af None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:17.413436 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a138b-38da-459a-8b4c-10b3ce0d63af None admin] PATH_INFO: `/v3/domains/default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:17.414721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127a138b-38da-459a-8b4c-10b3ce0d63af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:17.416333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127a138b-38da-459a-8b4c-10b3ce0d63af None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:17.418761 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 24/52] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:20:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:17.424125 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a138b-38da-459a-8b4c-10b3ce0d63af None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:17.449044 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:17.459388 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797dd0d-5368-4651-b0cb-6a58c1faaf90 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:17.493929 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61df5ac-d9cc-4564-aec5-41a54abba3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:17.494609 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61df5ac-d9cc-4564-aec5-41a54abba3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:17.494910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61df5ac-d9cc-4564-aec5-41a54abba3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:17.495197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61df5ac-d9cc-4564-aec5-41a54abba3e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:17.495976 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61df5ac-d9cc-4564-aec5-41a54abba3e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:17.496297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61df5ac-d9cc-4564-aec5-41a54abba3e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:17.497747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61df5ac-d9cc-4564-aec5-41a54abba3e7 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:17.506882 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 29/53] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:17.512714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61df5ac-d9cc-4564-aec5-41a54abba3e7 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:17.526084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a138b-38da-459a-8b4c-10b3ce0d63af None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:17.560936 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3f1ba-662e-4bcf-a14b-bfbb05573e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:17.561563 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3f1ba-662e-4bcf-a14b-bfbb05573e2f None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:17.561860 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3f1ba-662e-4bcf-a14b-bfbb05573e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:17.561972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:17.562171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3f1ba-662e-4bcf-a14b-bfbb05573e2f None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:17.563231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3f1ba-662e-4bcf-a14b-bfbb05573e2f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:17.564696 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3f1ba-662e-4bcf-a14b-bfbb05573e2f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:17.591320 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 25/54] 10.222.0.40 () {60 vars in 1140 bytes} [Thu Jun 16 17:20:17 2022] POST /identity/v3/projects => generated 298 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:20:17.616782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61df5ac-d9cc-4564-aec5-41a54abba3e7 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:17.653706 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f5429-8443-4cb7-bfdb-e258e59ae87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:17.654615 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f5429-8443-4cb7-bfdb-e258e59ae87f None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:17.654904 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f5429-8443-4cb7-bfdb-e258e59ae87f None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:17.655186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f5429-8443-4cb7-bfdb-e258e59ae87f None admin] PATH_INFO: `/v3/users/eac0995e068547e3890aa6066bdecd84` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:17.656687 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0f5429-8443-4cb7-bfdb-e258e59ae87f None admin] RBAC: Authorizing `identity:get_user(user_id=eac0995e068547e3890aa6066bdecd84)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:17.658186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0f5429-8443-4cb7-bfdb-e258e59ae87f None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:17.660872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 30/55] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:17 2022] GET /identity/v3/users/eac0995e068547e3890aa6066bdecd84 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:17.666307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3f1ba-662e-4bcf-a14b-bfbb05573e2f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:17.749348 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:17.778236 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3f1ba-662e-4bcf-a14b-bfbb05573e2f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:17.814326 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35dc45f-dc17-4e65-8ee5-362cb5beee94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:17.815174 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35dc45f-dc17-4e65-8ee5-362cb5beee94 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:17.815856 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35dc45f-dc17-4e65-8ee5-362cb5beee94 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:17.816146 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35dc45f-dc17-4e65-8ee5-362cb5beee94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:17.817383 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35dc45f-dc17-4e65-8ee5-362cb5beee94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:17.818879 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35dc45f-dc17-4e65-8ee5-362cb5beee94 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:17.820527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 26/56] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:20:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:17.826357 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f5429-8443-4cb7-bfdb-e258e59ae87f None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:17.928123 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f5429-8443-4cb7-bfdb-e258e59ae87f None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:17.944974 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:17.961922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513dae8e-b84e-4ab7-92bf-336f2df5b608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:17.962599 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dae8e-b84e-4ab7-92bf-336f2df5b608 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:17.962898 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dae8e-b84e-4ab7-92bf-336f2df5b608 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:17.963190 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dae8e-b84e-4ab7-92bf-336f2df5b608 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:17.964040 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513dae8e-b84e-4ab7-92bf-336f2df5b608 None admin] RBAC: Adding query filter params (role.id=25a16d8ea4084e9eacdebb7a4dcd2337, scope.domain.id=default, user.id=eac0995e068547e3890aa6066bdecd84) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:17.964365 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513dae8e-b84e-4ab7-92bf-336f2df5b608 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:17.965869 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513dae8e-b84e-4ab7-92bf-336f2df5b608 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:17.975314 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 31/57] 10.222.0.40 () {56 vars in 1322 bytes} [Thu Jun 16 17:20:17 2022] GET /identity/v3/role_assignments?user.id=eac0995e068547e3890aa6066bdecd84&scope.domain.id=default&role.id=25a16d8ea4084e9eacdebb7a4dcd2337 => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:19.723147 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ada57f-1960-42cb-ae1c-35cca4b31a82 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:19.723668 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ada57f-1960-42cb-ae1c-35cca4b31a82 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:19.723781 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ada57f-1960-42cb-ae1c-35cca4b31a82 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:19.725284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 27/58] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:19.730346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875a2f5-45bb-4f0e-894c-40d913fcf9cc None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:19.730760 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875a2f5-45bb-4f0e-894c-40d913fcf9cc None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:19.730937 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875a2f5-45bb-4f0e-894c-40d913fcf9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:19.747701 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-0875a2f5-45bb-4f0e-894c-40d913fcf9cc None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:19.748910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0875a2f5-45bb-4f0e-894c-40d913fcf9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:19.819655 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 32/59] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:20:19.844698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ada57f-1960-42cb-ae1c-35cca4b31a82 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:19.944331 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ada57f-1960-42cb-ae1c-35cca4b31a82 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:19.976932 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8142529-ff2b-4f48-bf21-ba0ef2f116f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:19.977691 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8142529-ff2b-4f48-bf21-ba0ef2f116f3 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:19.978031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8142529-ff2b-4f48-bf21-ba0ef2f116f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:19.978292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8142529-ff2b-4f48-bf21-ba0ef2f116f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:19.979466 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8142529-ff2b-4f48-bf21-ba0ef2f116f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:19.980904 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8142529-ff2b-4f48-bf21-ba0ef2f116f3 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:19.982532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 28/60] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:20:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:19.988058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875a2f5-45bb-4f0e-894c-40d913fcf9cc None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:20.085710 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875a2f5-45bb-4f0e-894c-40d913fcf9cc None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:20.099614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:20.120829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce49412-16ac-41e9-a89a-85ea2e169f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:20.121534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce49412-16ac-41e9-a89a-85ea2e169f93 None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:20.121833 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce49412-16ac-41e9-a89a-85ea2e169f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:20.122125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce49412-16ac-41e9-a89a-85ea2e169f93 None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:20.122846 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce49412-16ac-41e9-a89a-85ea2e169f93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:20.124329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce49412-16ac-41e9-a89a-85ea2e169f93 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:20.151114 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 33/61] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 17:20:19 2022] POST /identity/v3/users => generated 275 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:20:23.615117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adb632-13aa-44b2-a645-f75d612eabb5 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:23.615117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adb632-13aa-44b2-a645-f75d612eabb5 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:23.615117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adb632-13aa-44b2-a645-f75d612eabb5 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:23.615117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 29/62] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:23.621058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3b518-d107-4f79-8967-b65df6d57528 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:23.621603 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3b518-d107-4f79-8967-b65df6d57528 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:23.621698 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3b518-d107-4f79-8967-b65df6d57528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:23.643184 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-f5c3b518-d107-4f79-8967-b65df6d57528 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:23.644481 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-f5c3b518-d107-4f79-8967-b65df6d57528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:23.662864 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e4017-eaa5-41c2-b5c3-a3ee32db8f49 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:23.663484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e4017-eaa5-41c2-b5c3-a3ee32db8f49 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:23.664085 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e4017-eaa5-41c2-b5c3-a3ee32db8f49 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:23.665742 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 30/63] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:23.672085 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada594b7-900e-4dba-9c70-fd8187e8dedf None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:23.672613 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada594b7-900e-4dba-9c70-fd8187e8dedf None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:23.673119 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada594b7-900e-4dba-9c70-fd8187e8dedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:23.703438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-ada594b7-900e-4dba-9c70-fd8187e8dedf None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:23.704698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ada594b7-900e-4dba-9c70-fd8187e8dedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:23.713698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:23.773801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 34/64] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:23.784256 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1aeea-bbdb-4555-9286-48350dc0f68b None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:23.784623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1aeea-bbdb-4555-9286-48350dc0f68b None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:23.784839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1aeea-bbdb-4555-9286-48350dc0f68b None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:23.789078 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 35/65] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:23.795722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4960b-3ae2-41b7-bc25-c163496d4212 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:23.796240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4960b-3ae2-41b7-bc25-c163496d4212 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:23.796670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4960b-3ae2-41b7-bc25-c163496d4212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:23.828142 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-afa4960b-3ae2-41b7-bc25-c163496d4212 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:23.829693 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-afa4960b-3ae2-41b7-bc25-c163496d4212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:23.838667 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 31/66] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:20:23.839786 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada594b7-900e-4dba-9c70-fd8187e8dedf None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:23.910053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 36/67] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:23.912033 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a943483-c7c6-473d-8978-40b5911f6aa1 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:23.912559 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a943483-c7c6-473d-8978-40b5911f6aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:23.912854 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a943483-c7c6-473d-8978-40b5911f6aa1 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:23.914528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 37/68] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:23.919974 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a943483-c7c6-473d-8978-40b5911f6aa1 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:23.952216 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada594b7-900e-4dba-9c70-fd8187e8dedf None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:23.952405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:23.988970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2663aa-9471-4760-a819-a956764a84cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:23.989815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2663aa-9471-4760-a819-a956764a84cc None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:23.990103 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2663aa-9471-4760-a819-a956764a84cc None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:23.990384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2663aa-9471-4760-a819-a956764a84cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:23.999285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:24.002352 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2663aa-9471-4760-a819-a956764a84cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:24.011199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-dd2663aa-9471-4760-a819-a956764a84cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:24.011199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:24.011199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:24.011199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:24.011199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:24.011199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:24.011199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:24.011199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:24.011199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:24.011199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:24.011199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:24.011199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:24.011199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:24.011199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:24.011199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:24.011199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:24.012123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:24.012123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:24.012123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:24.012123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:24.012123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:24.012123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:24.012123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:24.012123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:24.012123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:24.012123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:24.012123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:24.012123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:24.012123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:24.012123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:24.012123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:24.012123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:24.012940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:24.012940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:24.012940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:24.012940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:24.012940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:24.012940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:24.012940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:24.012940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:24.012940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:24.012940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:24.012940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:24.012940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:20:24.012940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 32/69] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:24.013637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7a1b2-78e2-4945-89d1-09d17b53249c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:24.014129 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7a1b2-78e2-4945-89d1-09d17b53249c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:24.014243 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7a1b2-78e2-4945-89d1-09d17b53249c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:24.031035 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e7d7a1b2-78e2-4945-89d1-09d17b53249c None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:24.032240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e7d7a1b2-78e2-4945-89d1-09d17b53249c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:24.043039 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a943483-c7c6-473d-8978-40b5911f6aa1 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:24.078910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cff15b-3466-451f-83e7-cbcedf5d5ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:24.079716 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cff15b-3466-451f-83e7-cbcedf5d5ccb None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:24.080013 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cff15b-3466-451f-83e7-cbcedf5d5ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:24.080300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cff15b-3466-451f-83e7-cbcedf5d5ccb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:24.091662 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cff15b-3466-451f-83e7-cbcedf5d5ccb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:24.099931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-58cff15b-3466-451f-83e7-cbcedf5d5ccb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:20:24.099931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:24.099931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:24.099931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:24.099931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:24.099931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:24.099931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:24.099931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:24.099931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:24.099931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:24.099931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:24.099931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:24.099931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:24.099931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:24.099931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:24.099931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:24.100829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:24.100829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:24.100829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:24.100829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:24.100829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:24.100829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:24.100829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:24.100829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:24.100829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:24.100829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:24.100829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:24.100829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:24.100829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:24.100829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:24.100829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:24.100829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:24.101661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:24.101661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:24.101661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:24.101661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:24.101661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:24.101661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:24.101661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:24.101661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:24.101661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:24.101661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:24.101661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:20:24.101661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:20:24.101661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 38/70] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:20:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:24.102370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cff15b-3466-451f-83e7-cbcedf5d5ccb None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:24.105602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 33/71] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:24.107245 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699cd20-c68b-4b2b-9806-8ee12892b4e4 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:24.107547 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699cd20-c68b-4b2b-9806-8ee12892b4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:24.107841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699cd20-c68b-4b2b-9806-8ee12892b4e4 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:24.109223 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 34/72] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:24.109940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699cd20-c68b-4b2b-9806-8ee12892b4e4 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:24.197609 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:24.210222 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cff15b-3466-451f-83e7-cbcedf5d5ccb None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:24.223727 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699cd20-c68b-4b2b-9806-8ee12892b4e4 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:24.248823 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec55040-ed04-487a-bb33-208c85926476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:24.249911 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec55040-ed04-487a-bb33-208c85926476 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:24.250264 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec55040-ed04-487a-bb33-208c85926476 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:24.250686 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec55040-ed04-487a-bb33-208c85926476 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:24.255451 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:24.261840 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec55040-ed04-487a-bb33-208c85926476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:24.262258 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d162c-3e16-465e-93d9-4d8e495cfc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:24.262939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d162c-3e16-465e-93d9-4d8e495cfc37 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:24.263231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d162c-3e16-465e-93d9-4d8e495cfc37 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:24.263516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d162c-3e16-465e-93d9-4d8e495cfc37 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:24.264286 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d162c-3e16-465e-93d9-4d8e495cfc37 None admin] RBAC: Adding query filter params (name=member) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:24.264607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d162c-3e16-465e-93d9-4d8e495cfc37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:24.266058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d162c-3e16-465e-93d9-4d8e495cfc37 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:24.270747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-1ec55040-ed04-487a-bb33-208c85926476 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:20:24.270747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:24.270747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:24.270747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:24.270747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:24.270747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:24.270747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:24.270747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:24.270747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:24.270747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:24.270747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:24.270747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:24.270747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:24.270747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:24.270747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:24.270747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:24.271632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:24.271632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:24.271632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:24.271632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:24.271632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:24.271632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:24.271632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:24.271632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:24.271632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:24.271632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:24.271632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:24.271632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:24.271632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:24.271632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:24.271632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:24.271632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:24.272449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:24.272449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:24.272449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:24.272449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:24.272449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:24.272449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:24.272449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:24.272449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:24.272449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:24.272449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:24.272449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:20:24.272449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:20:24.272449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 39/73] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:20:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:24.273724 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec55040-ed04-487a-bb33-208c85926476 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:24.276980 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 35/74] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:20:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:24.277895 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579eeeb-a0f6-48ee-b8cc-2587ebdb6129 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:24.278193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579eeeb-a0f6-48ee-b8cc-2587ebdb6129 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:24.278477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579eeeb-a0f6-48ee-b8cc-2587ebdb6129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:24.295900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-5579eeeb-a0f6-48ee-b8cc-2587ebdb6129 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:24.297268 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5579eeeb-a0f6-48ee-b8cc-2587ebdb6129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:24.364676 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 36/75] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:24.365357 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579eeeb-a0f6-48ee-b8cc-2587ebdb6129 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:24.374184 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec55040-ed04-487a-bb33-208c85926476 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:24.408479 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070adad-82b9-4d17-992d-b54389a3e9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:24.409847 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070adad-82b9-4d17-992d-b54389a3e9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:24.410146 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070adad-82b9-4d17-992d-b54389a3e9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:24.410425 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070adad-82b9-4d17-992d-b54389a3e9c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:24.411213 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070adad-82b9-4d17-992d-b54389a3e9c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:24.411528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070adad-82b9-4d17-992d-b54389a3e9c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:24.412917 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070adad-82b9-4d17-992d-b54389a3e9c3 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:24.421121 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 40/76] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:24.421734 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070adad-82b9-4d17-992d-b54389a3e9c3 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:24.434988 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:24.471160 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579eeeb-a0f6-48ee-b8cc-2587ebdb6129 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:24.494572 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:24.508838 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e2d41-8581-4a92-9c02-51b4bdb3a47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:24.509941 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e2d41-8581-4a92-9c02-51b4bdb3a47b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:24.510236 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e2d41-8581-4a92-9c02-51b4bdb3a47b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:24.510524 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e2d41-8581-4a92-9c02-51b4bdb3a47b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:24.521799 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e2d41-8581-4a92-9c02-51b4bdb3a47b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:24.526949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070adad-82b9-4d17-992d-b54389a3e9c3 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:24.529681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-9b5e2d41-8581-4a92-9c02-51b4bdb3a47b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:24.529681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:24.529681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:24.529681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:24.529681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:24.529681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:24.529681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:24.529681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:24.529681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:24.529681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:24.529681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:24.529681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:24.529681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:24.529681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:24.529681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:24.529681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:24.530726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:24.530726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:24.530726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:24.530726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:24.530726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:24.530726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:24.530726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:24.530726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:24.530726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:24.530726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:24.530726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:24.530726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:24.530726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:24.530726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:24.530726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:24.530726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:24.531515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:24.531515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:24.531515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:24.531515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:24.531515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:24.531515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:24.531515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:24.531515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:24.531515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:24.531515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:24.531515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:24.531515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:20:24.531515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 37/77] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:24.537298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e2d41-8581-4a92-9c02-51b4bdb3a47b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:24.561471 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a519eb-fc75-44e1-a5b3-3af57a3d230e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:24.562159 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a519eb-fc75-44e1-a5b3-3af57a3d230e None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:24.562457 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a519eb-fc75-44e1-a5b3-3af57a3d230e None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:24.562749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a519eb-fc75-44e1-a5b3-3af57a3d230e None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:24.563520 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a519eb-fc75-44e1-a5b3-3af57a3d230e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:24.563845 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a519eb-fc75-44e1-a5b3-3af57a3d230e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:24.565891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a519eb-fc75-44e1-a5b3-3af57a3d230e None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:24.573839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 41/78] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:20:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:24.574819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a519eb-fc75-44e1-a5b3-3af57a3d230e None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:24.641758 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e2d41-8581-4a92-9c02-51b4bdb3a47b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:24.643810 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:24.674110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a519eb-fc75-44e1-a5b3-3af57a3d230e None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:24.676087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ed0da-565b-4f36-97d3-1625dcf573c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:24.676976 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ed0da-565b-4f36-97d3-1625dcf573c9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:24.677297 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ed0da-565b-4f36-97d3-1625dcf573c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:24.677588 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ed0da-565b-4f36-97d3-1625dcf573c9 None admin] PATH_INFO: `/v3/users/ed0016d7b075461eb857ad7527a36809` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:24.691267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ed0da-565b-4f36-97d3-1625dcf573c9 None admin] RBAC: Authorizing `identity:get_user(user_id=ed0016d7b075461eb857ad7527a36809)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:24.692775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ed0da-565b-4f36-97d3-1625dcf573c9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:24.694568 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 38/79] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:24 2022] GET /identity/v3/users/ed0016d7b075461eb857ad7527a36809 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:24.696535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ed0da-565b-4f36-97d3-1625dcf573c9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:24.709995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd286f4b-427b-4143-a3fa-85aa13db24c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:24.710810 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd286f4b-427b-4143-a3fa-85aa13db24c2 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:24.711106 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd286f4b-427b-4143-a3fa-85aa13db24c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:24.711397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd286f4b-427b-4143-a3fa-85aa13db24c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:24.712607 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd286f4b-427b-4143-a3fa-85aa13db24c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:24.714102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd286f4b-427b-4143-a3fa-85aa13db24c2 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:24.715758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 42/80] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:20:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:24.719691 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd286f4b-427b-4143-a3fa-85aa13db24c2 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:24.739206 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:24.796628 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ed0da-565b-4f36-97d3-1625dcf573c9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:24.823930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd286f4b-427b-4143-a3fa-85aa13db24c2 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:24.831060 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daac60c-ef3c-4479-9d7d-22cda360140f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:24.832034 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daac60c-ef3c-4479-9d7d-22cda360140f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:24.832324 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daac60c-ef3c-4479-9d7d-22cda360140f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:24.832612 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daac60c-ef3c-4479-9d7d-22cda360140f None admin] PATH_INFO: `/v3/users/eac0995e068547e3890aa6066bdecd84` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:24.833872 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daac60c-ef3c-4479-9d7d-22cda360140f None admin] RBAC: Authorizing `identity:get_user(user_id=eac0995e068547e3890aa6066bdecd84)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:24.835342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daac60c-ef3c-4479-9d7d-22cda360140f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:24.836999 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 39/81] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:24 2022] GET /identity/v3/users/eac0995e068547e3890aa6066bdecd84 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:24.842006 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daac60c-ef3c-4479-9d7d-22cda360140f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:24.858249 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659f51c-158b-4f04-bffe-35804edf50b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:24.858934 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659f51c-158b-4f04-bffe-35804edf50b8 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:24.859233 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659f51c-158b-4f04-bffe-35804edf50b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:24.859522 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659f51c-158b-4f04-bffe-35804edf50b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:24.860294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659f51c-158b-4f04-bffe-35804edf50b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:24.860615 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659f51c-158b-4f04-bffe-35804edf50b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:24.862584 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659f51c-158b-4f04-bffe-35804edf50b8 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:24.870913 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 43/82] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:20:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:24.876375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659f51c-158b-4f04-bffe-35804edf50b8 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:24.879683 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:24.943690 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daac60c-ef3c-4479-9d7d-22cda360140f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:24.953603 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:24.976512 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659f51c-158b-4f04-bffe-35804edf50b8 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:24.979440 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f2ec3-049e-42ee-a685-449391a3ea38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:24.980328 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f2ec3-049e-42ee-a685-449391a3ea38 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:24.980620 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f2ec3-049e-42ee-a685-449391a3ea38 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:24.980904 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f2ec3-049e-42ee-a685-449391a3ea38 None admin] PATH_INFO: `/v3/users/ed0016d7b075461eb857ad7527a36809` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:24.982183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f2ec3-049e-42ee-a685-449391a3ea38 None admin] RBAC: Authorizing `identity:get_user(user_id=ed0016d7b075461eb857ad7527a36809)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:24.983648 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f2ec3-049e-42ee-a685-449391a3ea38 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:24.985318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 40/83] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:24 2022] GET /identity/v3/users/ed0016d7b075461eb857ad7527a36809 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:24.986064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f2ec3-049e-42ee-a685-449391a3ea38 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:25.011466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c339eee-1b37-4424-b17f-80a88a16d04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:25.012058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c339eee-1b37-4424-b17f-80a88a16d04e None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:25.012360 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c339eee-1b37-4424-b17f-80a88a16d04e None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:25.012649 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c339eee-1b37-4424-b17f-80a88a16d04e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:25.081184 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c339eee-1b37-4424-b17f-80a88a16d04e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:25.082483 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c339eee-1b37-4424-b17f-80a88a16d04e None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:25.084310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:25.088800 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f2ec3-049e-42ee-a685-449391a3ea38 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:25.132669 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de14fe-666d-42a7-9857-018bf6a485c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:25.133406 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de14fe-666d-42a7-9857-018bf6a485c5 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:25.133704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de14fe-666d-42a7-9857-018bf6a485c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:25.133972 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de14fe-666d-42a7-9857-018bf6a485c5 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:25.134968 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7de14fe-666d-42a7-9857-018bf6a485c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:25.136433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7de14fe-666d-42a7-9857-018bf6a485c5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:25.163774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 41/84] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 17:20:24 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:20:25.164424 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de14fe-666d-42a7-9857-018bf6a485c5 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:25.201530 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:25.211551 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 44/85] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:20:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:20:25.212761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c339eee-1b37-4424-b17f-80a88a16d04e None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:25.292572 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de14fe-666d-42a7-9857-018bf6a485c5 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:25.333795 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8475af-b4d2-4acd-b8aa-ba83850aa389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:25.334529 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8475af-b4d2-4acd-b8aa-ba83850aa389 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:25.334834 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8475af-b4d2-4acd-b8aa-ba83850aa389 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:25.335126 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8475af-b4d2-4acd-b8aa-ba83850aa389 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:25.342236 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c339eee-1b37-4424-b17f-80a88a16d04e None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:25.344062 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:25.382260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3c63a-8ea9-4de4-944d-aeceb3f19eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:25.383164 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3c63a-8ea9-4de4-944d-aeceb3f19eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:25.383456 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3c63a-8ea9-4de4-944d-aeceb3f19eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:25.383744 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3c63a-8ea9-4de4-944d-aeceb3f19eb1 None admin] PATH_INFO: `/v3/users/ed0016d7b075461eb857ad7527a36809` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:25.385354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c3c63a-8ea9-4de4-944d-aeceb3f19eb1 None admin] RBAC: Authorizing `identity:get_user(user_id=ed0016d7b075461eb857ad7527a36809)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:25.387924 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c3c63a-8ea9-4de4-944d-aeceb3f19eb1 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:25.392148 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 45/86] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:25 2022] GET /identity/v3/users/ed0016d7b075461eb857ad7527a36809 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:25.392148 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3c63a-8ea9-4de4-944d-aeceb3f19eb1 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:25.428644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8475af-b4d2-4acd-b8aa-ba83850aa389 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:25.430474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8475af-b4d2-4acd-b8aa-ba83850aa389 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:25.475885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:25.536839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3c63a-8ea9-4de4-944d-aeceb3f19eb1 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:25.561901 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 42/87] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:20:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:20:25.562690 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8475af-b4d2-4acd-b8aa-ba83850aa389 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:25.580404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a06f2-5f5a-4b12-ac5b-6135e2fab05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:25.581956 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a06f2-5f5a-4b12-ac5b-6135e2fab05d None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:25.582064 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a06f2-5f5a-4b12-ac5b-6135e2fab05d None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:25.585181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a06f2-5f5a-4b12-ac5b-6135e2fab05d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:25.626332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:25.664803 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a06f2-5f5a-4b12-ac5b-6135e2fab05d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:25.666537 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a06f2-5f5a-4b12-ac5b-6135e2fab05d None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:25.669251 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8475af-b4d2-4acd-b8aa-ba83850aa389 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:25.700734 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:25.707127 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48668e9-4eab-4304-9834-de0467f8d071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:25.708100 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48668e9-4eab-4304-9834-de0467f8d071 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:25.708465 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48668e9-4eab-4304-9834-de0467f8d071 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:25.708824 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48668e9-4eab-4304-9834-de0467f8d071 None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:25.710221 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48668e9-4eab-4304-9834-de0467f8d071 None admin] RBAC: Authorizing `identity:get_project(project_id=8a3d0e211dd942aaa39eed1730d17e92)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:25.711777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48668e9-4eab-4304-9834-de0467f8d071 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:25.713803 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 43/88] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:25 2022] GET /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:25.714522 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48668e9-4eab-4304-9834-de0467f8d071 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:25.790685 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 46/89] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:20:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:20:25.797563 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a06f2-5f5a-4b12-ac5b-6135e2fab05d None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:25.823453 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48668e9-4eab-4304-9834-de0467f8d071 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:25.862629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147ed86-a643-440a-a045-992e033788e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:25.863471 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147ed86-a643-440a-a045-992e033788e8 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:25.863862 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147ed86-a643-440a-a045-992e033788e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:25.864265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147ed86-a643-440a-a045-992e033788e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:25.895766 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:25.936045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:25.938669 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147ed86-a643-440a-a045-992e033788e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:25.939095 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a06f2-5f5a-4b12-ac5b-6135e2fab05d None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:25.941290 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147ed86-a643-440a-a045-992e033788e8 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:26.004155 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90772c-f435-4579-b0ea-bf3b5f17af73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:26.005988 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90772c-f435-4579-b0ea-bf3b5f17af73 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:26.006476 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90772c-f435-4579-b0ea-bf3b5f17af73 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:26.007409 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90772c-f435-4579-b0ea-bf3b5f17af73 None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:26.010665 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90772c-f435-4579-b0ea-bf3b5f17af73 None admin] RBAC: Authorizing `identity:get_project(project_id=8a3d0e211dd942aaa39eed1730d17e92)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:26.014052 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90772c-f435-4579-b0ea-bf3b5f17af73 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:26.018061 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 47/90] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:25 2022] GET /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:26.027820 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90772c-f435-4579-b0ea-bf3b5f17af73 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:26.057280 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 44/91] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:20:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:20:26.058314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147ed86-a643-440a-a045-992e033788e8 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:26.170505 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:26.174103 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147ed86-a643-440a-a045-992e033788e8 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:26.198253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90772c-f435-4579-b0ea-bf3b5f17af73 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:26.212864 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78e126-0542-49ab-bcc3-29d92ffcee90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:26.213794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78e126-0542-49ab-bcc3-29d92ffcee90 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:26.214085 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78e126-0542-49ab-bcc3-29d92ffcee90 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:26.214368 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78e126-0542-49ab-bcc3-29d92ffcee90 None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:26.215605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78e126-0542-49ab-bcc3-29d92ffcee90 None admin] RBAC: Authorizing `identity:get_project(project_id=8a3d0e211dd942aaa39eed1730d17e92)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:26.217067 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78e126-0542-49ab-bcc3-29d92ffcee90 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:26.218830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 45/92] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:26 2022] GET /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:26.219605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78e126-0542-49ab-bcc3-29d92ffcee90 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:26.260383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4207030-b4a9-49be-abe9-802c38f7a05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:26.261341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4207030-b4a9-49be-abe9-802c38f7a05e None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:26.261775 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4207030-b4a9-49be-abe9-802c38f7a05e None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:26.262267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4207030-b4a9-49be-abe9-802c38f7a05e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:26.263262 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4207030-b4a9-49be-abe9-802c38f7a05e None admin] RBAC: Adding query filter params (role.id=483146fdc4ac47cf97c04a9f66c091b4, scope.project.id=8a3d0e211dd942aaa39eed1730d17e92, user.id=ed0016d7b075461eb857ad7527a36809) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:26.264168 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4207030-b4a9-49be-abe9-802c38f7a05e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:26.265818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4207030-b4a9-49be-abe9-802c38f7a05e None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:26.274259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 48/93] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:20:26 2022] GET /identity/v3/role_assignments?user.id=ed0016d7b075461eb857ad7527a36809&scope.project.id=8a3d0e211dd942aaa39eed1730d17e92&role.id=483146fdc4ac47cf97c04a9f66c091b4 => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:26.275391 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4207030-b4a9-49be-abe9-802c38f7a05e None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:26.278215 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:26.436352 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78e126-0542-49ab-bcc3-29d92ffcee90 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:26.478657 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4207030-b4a9-49be-abe9-802c38f7a05e None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:26.526997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebe924-05a8-4690-be3d-d706e8c9f65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:26.527690 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebe924-05a8-4690-be3d-d706e8c9f65d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:26.527981 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebe924-05a8-4690-be3d-d706e8c9f65d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:26.528264 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebe924-05a8-4690-be3d-d706e8c9f65d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:26.529162 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebe924-05a8-4690-be3d-d706e8c9f65d None admin] RBAC: Adding query filter params (role.id=25a16d8ea4084e9eacdebb7a4dcd2337, scope.project.id=8a3d0e211dd942aaa39eed1730d17e92, user.id=eac0995e068547e3890aa6066bdecd84) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:26.529488 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebe924-05a8-4690-be3d-d706e8c9f65d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:26.530999 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebe924-05a8-4690-be3d-d706e8c9f65d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:26.546267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 46/94] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:20:26 2022] GET /identity/v3/role_assignments?user.id=eac0995e068547e3890aa6066bdecd84&scope.project.id=8a3d0e211dd942aaa39eed1730d17e92&role.id=25a16d8ea4084e9eacdebb7a4dcd2337 => generated 259 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:26.547172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebe924-05a8-4690-be3d-d706e8c9f65d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:26.569002 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10336c-c522-487d-b0e9-be8d9615cb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:26.570007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10336c-c522-487d-b0e9-be8d9615cb76 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:26.570306 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10336c-c522-487d-b0e9-be8d9615cb76 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:26.570600 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10336c-c522-487d-b0e9-be8d9615cb76 None admin] PATH_INFO: `/v3/projects/ca04884049304c19bdf4ab8b556ac8db` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:26.572131 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10336c-c522-487d-b0e9-be8d9615cb76 None admin] RBAC: Authorizing `identity:get_project(project_id=ca04884049304c19bdf4ab8b556ac8db)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:26.573689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10336c-c522-487d-b0e9-be8d9615cb76 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:26.576975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 49/95] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:26 2022] GET /identity/v3/projects/ca04884049304c19bdf4ab8b556ac8db => generated 312 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:26.586166 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10336c-c522-487d-b0e9-be8d9615cb76 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:26.631445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:26.762718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:26.792544 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebe924-05a8-4690-be3d-d706e8c9f65d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:26.817907 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10336c-c522-487d-b0e9-be8d9615cb76 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:26.853523 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e32d1-0106-4a21-a61a-2c2b83fc6729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:26.854780 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e32d1-0106-4a21-a61a-2c2b83fc6729 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:26.857924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e32d1-0106-4a21-a61a-2c2b83fc6729 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:26.858444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e32d1-0106-4a21-a61a-2c2b83fc6729 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:26.859512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e32d1-0106-4a21-a61a-2c2b83fc6729 None admin] RBAC: Adding query filter params (role.id=1423a24997e840608b6c522958481fde, scope.project.id=8a3d0e211dd942aaa39eed1730d17e92, user.id=ed0016d7b075461eb857ad7527a36809) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:26.860037 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e32d1-0106-4a21-a61a-2c2b83fc6729 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:26.862013 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e32d1-0106-4a21-a61a-2c2b83fc6729 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:26.877282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 47/96] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:20:26 2022] GET /identity/v3/role_assignments?user.id=ed0016d7b075461eb857ad7527a36809&scope.project.id=8a3d0e211dd942aaa39eed1730d17e92&role.id=1423a24997e840608b6c522958481fde => generated 259 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:26.892804 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9258e0dd-f309-432d-b521-6041b808ff69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:26.895041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258e0dd-f309-432d-b521-6041b808ff69 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:26.895746 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258e0dd-f309-432d-b521-6041b808ff69 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:26.896305 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258e0dd-f309-432d-b521-6041b808ff69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:26.899483 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9258e0dd-f309-432d-b521-6041b808ff69 None admin] RBAC: Adding query filter params (role.id=483146fdc4ac47cf97c04a9f66c091b4, scope.project.id=ca04884049304c19bdf4ab8b556ac8db, user.id=ed0016d7b075461eb857ad7527a36809) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:26.900031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9258e0dd-f309-432d-b521-6041b808ff69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:26.901906 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9258e0dd-f309-432d-b521-6041b808ff69 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:26.913320 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 50/97] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:20:26 2022] GET /identity/v3/role_assignments?user.id=ed0016d7b075461eb857ad7527a36809&scope.project.id=ca04884049304c19bdf4ab8b556ac8db&role.id=483146fdc4ac47cf97c04a9f66c091b4 => generated 259 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:27.793065 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da3cbc-fef9-4b11-bca8-70bdae28c895 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:27.793609 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da3cbc-fef9-4b11-bca8-70bdae28c895 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:27.793684 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da3cbc-fef9-4b11-bca8-70bdae28c895 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:27.795232 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 48/98] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:27.800270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7d05e-70b7-45ea-99e1-29523f52af98 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:27.800680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7d05e-70b7-45ea-99e1-29523f52af98 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:27.800847 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7d05e-70b7-45ea-99e1-29523f52af98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:27.817854 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-e4e7d05e-70b7-45ea-99e1-29523f52af98 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:27.818958 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-e4e7d05e-70b7-45ea-99e1-29523f52af98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:27.890923 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 51/99] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:27.915252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da3cbc-fef9-4b11-bca8-70bdae28c895 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:27.948771 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7faa0cb-18d1-4571-8331-63b670b22721 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:27.949066 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7faa0cb-18d1-4571-8331-63b670b22721 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:27.949370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7faa0cb-18d1-4571-8331-63b670b22721 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:27.950978 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 52/100] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:27.962986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f33fe-3800-4553-8a5f-666155b6c3f8 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:27.963282 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f33fe-3800-4553-8a5f-666155b6c3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:27.963566 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f33fe-3800-4553-8a5f-666155b6c3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:27.987097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-860f33fe-3800-4553-8a5f-666155b6c3f8 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:27.988302 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-860f33fe-3800-4553-8a5f-666155b6c3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:28.085714 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:28.095949 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da3cbc-fef9-4b11-bca8-70bdae28c895 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:28.101070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:28.116943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 53/101] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:20:28.145824 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2ff72-a16e-41e6-92b6-12607bccade6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:28.146720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2ff72-a16e-41e6-92b6-12607bccade6 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:28.147018 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2ff72-a16e-41e6-92b6-12607bccade6 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:28.147304 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2ff72-a16e-41e6-92b6-12607bccade6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:28.165080 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f33fe-3800-4553-8a5f-666155b6c3f8 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:28.184528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2ff72-a16e-41e6-92b6-12607bccade6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:28.193970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-0da2ff72-a16e-41e6-92b6-12607bccade6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:20:28.193970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:28.193970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:28.193970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:28.193970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:28.193970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:28.193970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:28.193970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:28.193970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:28.193970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:28.193970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:28.193970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:28.193970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:28.193970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:28.193970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:28.193970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:28.194880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:28.194880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:28.194880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:28.194880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:28.194880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:28.194880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:28.194880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:28.194880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:28.194880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:28.194880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:28.194880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:28.194880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:28.194880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:28.194880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:28.194880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:28.194880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:28.195674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:28.195674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:28.195674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:28.195674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:28.195674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:28.195674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:28.195674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:28.195674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:28.195674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:28.195674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:28.195674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:20:28.195674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:20:28.203841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 49/102] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:28.217394 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2ff72-a16e-41e6-92b6-12607bccade6 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:28.308709 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f33fe-3800-4553-8a5f-666155b6c3f8 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:28.345794 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ed52a-b21a-4f9c-bc5d-67fea2ea6e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:28.346629 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ed52a-b21a-4f9c-bc5d-67fea2ea6e23 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:28.346953 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ed52a-b21a-4f9c-bc5d-67fea2ea6e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:28.347285 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ed52a-b21a-4f9c-bc5d-67fea2ea6e23 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:28.348552 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ed52a-b21a-4f9c-bc5d-67fea2ea6e23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:28.350073 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ed52a-b21a-4f9c-bc5d-67fea2ea6e23 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:28.351969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 54/103] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:20:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:28.357140 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ed52a-b21a-4f9c-bc5d-67fea2ea6e23 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:28.421585 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2ff72-a16e-41e6-92b6-12607bccade6 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:28.464719 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84828a59-bef7-469c-8d11-14948d39c66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:28.465446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84828a59-bef7-469c-8d11-14948d39c66b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:28.465747 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84828a59-bef7-469c-8d11-14948d39c66b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:28.466039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84828a59-bef7-469c-8d11-14948d39c66b None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:28.467111 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84828a59-bef7-469c-8d11-14948d39c66b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:28.467433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84828a59-bef7-469c-8d11-14948d39c66b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:28.468847 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84828a59-bef7-469c-8d11-14948d39c66b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:28.491007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 50/104] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:20:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:28.496840 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84828a59-bef7-469c-8d11-14948d39c66b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:28.499726 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:28.536326 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:28.558522 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ed52a-b21a-4f9c-bc5d-67fea2ea6e23 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:28.593251 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a0dd8-f5a4-4ea3-90da-d72750980b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:28.593920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a0dd8-f5a4-4ea3-90da-d72750980b18 None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:28.594213 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a0dd8-f5a4-4ea3-90da-d72750980b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:28.594498 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a0dd8-f5a4-4ea3-90da-d72750980b18 None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:28.595216 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a0dd8-f5a4-4ea3-90da-d72750980b18 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:28.596762 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a0dd8-f5a4-4ea3-90da-d72750980b18 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:28.626226 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 55/105] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 17:20:28 2022] POST /identity/v3/projects => generated 302 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:20:28.678050 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84828a59-bef7-469c-8d11-14948d39c66b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:28.733820 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7b167-40b3-414c-9d64-8daea6ecfe6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:28.734730 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7b167-40b3-414c-9d64-8daea6ecfe6d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:28.735028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7b167-40b3-414c-9d64-8daea6ecfe6d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:28.735321 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7b167-40b3-414c-9d64-8daea6ecfe6d None admin] PATH_INFO: `/v3/users/a22703a1b8784cb888c43670d59fb559` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:28.751033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f7b167-40b3-414c-9d64-8daea6ecfe6d None admin] RBAC: Authorizing `identity:get_user(user_id=a22703a1b8784cb888c43670d59fb559)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:28.753268 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f7b167-40b3-414c-9d64-8daea6ecfe6d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:28.755483 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 51/106] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:28 2022] GET /identity/v3/users/a22703a1b8784cb888c43670d59fb559 => generated 289 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:28.761430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a0dd8-f5a4-4ea3-90da-d72750980b18 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:28.894297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a0dd8-f5a4-4ea3-90da-d72750980b18 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:28.913630 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:28.939600 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8237668-fc57-4915-98a9-5b8f7e0d2b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:28.940957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8237668-fc57-4915-98a9-5b8f7e0d2b99 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:28.941290 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8237668-fc57-4915-98a9-5b8f7e0d2b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:28.941639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8237668-fc57-4915-98a9-5b8f7e0d2b99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:29.034789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8237668-fc57-4915-98a9-5b8f7e0d2b99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:29.036305 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8237668-fc57-4915-98a9-5b8f7e0d2b99 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:29.172626 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 56/107] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:20:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:20:29.178632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7b167-40b3-414c-9d64-8daea6ecfe6d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:29.319606 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:29.362606 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7b167-40b3-414c-9d64-8daea6ecfe6d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:29.415675 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a6e30-6d8f-461b-937c-e4c71a9a5f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:29.417041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a6e30-6d8f-461b-937c-e4c71a9a5f81 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:29.417544 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a6e30-6d8f-461b-937c-e4c71a9a5f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:29.418004 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a6e30-6d8f-461b-937c-e4c71a9a5f81 None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:29.419619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a6e30-6d8f-461b-937c-e4c71a9a5f81 None admin] RBAC: Authorizing `identity:get_project(project_id=8a3d0e211dd942aaa39eed1730d17e92)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:29.421089 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a6e30-6d8f-461b-937c-e4c71a9a5f81 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:29.422904 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 52/108] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:29 2022] GET /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92 => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:29.429242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8237668-fc57-4915-98a9-5b8f7e0d2b99 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:29.444983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:29.560933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8237668-fc57-4915-98a9-5b8f7e0d2b99 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:29.610435 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9315074-df2f-47c9-bffd-6ff40108d592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:29.611510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9315074-df2f-47c9-bffd-6ff40108d592 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:29.612040 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9315074-df2f-47c9-bffd-6ff40108d592 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:29.612526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9315074-df2f-47c9-bffd-6ff40108d592 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:29.613625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9315074-df2f-47c9-bffd-6ff40108d592 None admin] RBAC: Adding query filter params (role.id=075fc2a9c4184b9db26b044f3d963233, scope.project.id=8a3d0e211dd942aaa39eed1730d17e92, user.id=a22703a1b8784cb888c43670d59fb559) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:29.614159 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9315074-df2f-47c9-bffd-6ff40108d592 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:29.615845 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9315074-df2f-47c9-bffd-6ff40108d592 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:29.624261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 57/109] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:20:29 2022] GET /identity/v3/role_assignments?user.id=a22703a1b8784cb888c43670d59fb559&scope.project.id=8a3d0e211dd942aaa39eed1730d17e92&role.id=075fc2a9c4184b9db26b044f3d963233 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:29.745918 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5275b-997e-469d-ae91-d97a3d87e707 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:29.746291 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5275b-997e-469d-ae91-d97a3d87e707 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:29.746730 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5275b-997e-469d-ae91-d97a3d87e707 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:29.749242 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 53/110] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:29.755830 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6410049-23f1-4497-8e48-524a0a49a083 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:29.756452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6410049-23f1-4497-8e48-524a0a49a083 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:29.756946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6410049-23f1-4497-8e48-524a0a49a083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:29.785616 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-f6410049-23f1-4497-8e48-524a0a49a083 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:29.787195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-f6410049-23f1-4497-8e48-524a0a49a083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:29.890780 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 58/111] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:20:29.926742 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5275b-997e-469d-ae91-d97a3d87e707 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:29.980266 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da5c4c-b668-4b0b-aa39-17dadaa3e291 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:29.981306 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da5c4c-b668-4b0b-aa39-17dadaa3e291 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:29.981792 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da5c4c-b668-4b0b-aa39-17dadaa3e291 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:29.983470 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 59/112] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:30.001907 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533464d-546c-4eab-97a7-64b7018e96a6 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:30.003073 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533464d-546c-4eab-97a7-64b7018e96a6 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:30.003563 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533464d-546c-4eab-97a7-64b7018e96a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:30.038595 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-9533464d-546c-4eab-97a7-64b7018e96a6 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:30.042391 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-9533464d-546c-4eab-97a7-64b7018e96a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:30.045441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:30.077982 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5275b-997e-469d-ae91-d97a3d87e707 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:30.149798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd349202-17f3-4ee2-9222-4e33f168dffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:30.150872 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd349202-17f3-4ee2-9222-4e33f168dffd None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:30.151347 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd349202-17f3-4ee2-9222-4e33f168dffd None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:30.151786 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd349202-17f3-4ee2-9222-4e33f168dffd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:30.164410 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd349202-17f3-4ee2-9222-4e33f168dffd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:30.172987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-fd349202-17f3-4ee2-9222-4e33f168dffd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:30.172987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:30.172987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:30.172987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:30.172987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:30.172987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:30.172987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:30.172987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:30.172987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:30.172987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:30.172987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:30.172987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:30.172987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:30.172987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:30.172987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:30.172987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:30.174687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:30.174687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:30.174687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:30.174687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:30.174687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:30.174687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:30.174687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:30.174687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:30.174687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:30.174687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:30.174687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:30.174687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:30.174687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:30.174687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:30.174687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:30.174687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:30.175841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:30.175841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:30.175841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:30.175841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:30.175841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:30.175841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:30.175841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:30.175841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:30.175841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:30.175841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:30.175841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:30.175841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:20:30.178290 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 54/113] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:30.184457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd349202-17f3-4ee2-9222-4e33f168dffd None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:30.193679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:30.216931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 60/114] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:20:30.219227 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2d40a-5ce0-479d-b26e-65affb3dd055 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:30.219735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2d40a-5ce0-479d-b26e-65affb3dd055 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:30.220225 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2d40a-5ce0-479d-b26e-65affb3dd055 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:30.222090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 61/115] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:30.231125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a2cc4-f6d7-41a6-97c1-7e44b6d6711a None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:30.231714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a2cc4-f6d7-41a6-97c1-7e44b6d6711a None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:30.232207 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a2cc4-f6d7-41a6-97c1-7e44b6d6711a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:30.261488 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-0f2a2cc4-f6d7-41a6-97c1-7e44b6d6711a None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:30.262662 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0f2a2cc4-f6d7-41a6-97c1-7e44b6d6711a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:30.352493 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 62/116] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:30.353406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a2cc4-f6d7-41a6-97c1-7e44b6d6711a None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:30.412193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd349202-17f3-4ee2-9222-4e33f168dffd None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:30.447660 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:30.462403 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb44923-e185-4a07-b3e1-ad006c32387b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:30.463153 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb44923-e185-4a07-b3e1-ad006c32387b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:30.463480 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb44923-e185-4a07-b3e1-ad006c32387b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:30.464018 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb44923-e185-4a07-b3e1-ad006c32387b None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:30.464805 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb44923-e185-4a07-b3e1-ad006c32387b None admin] RBAC: Adding query filter params (name=member) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:30.465127 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb44923-e185-4a07-b3e1-ad006c32387b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:30.467365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb44923-e185-4a07-b3e1-ad006c32387b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:30.483769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 55/117] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:20:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:30.483769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07a74b-66c2-4b4c-be56-efba3d0a595d None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:30.483769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07a74b-66c2-4b4c-be56-efba3d0a595d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:30.483769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07a74b-66c2-4b4c-be56-efba3d0a595d None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:30.483769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 56/118] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:30.483769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07a74b-66c2-4b4c-be56-efba3d0a595d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:30.494779 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a2cc4-f6d7-41a6-97c1-7e44b6d6711a None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:30.544056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb07df-86dc-4745-a91f-936795b7ff9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:30.544882 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb07df-86dc-4745-a91f-936795b7ff9c None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:30.545333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb07df-86dc-4745-a91f-936795b7ff9c None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:30.545958 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb07df-86dc-4745-a91f-936795b7ff9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:30.561373 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb07df-86dc-4745-a91f-936795b7ff9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:30.563545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:30.572374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-7bdb07df-86dc-4745-a91f-936795b7ff9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:20:30.572374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:30.572374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:30.572374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:30.572374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:30.572374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:30.572374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:30.572374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:30.572374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:30.572374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:30.572374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:30.572374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:30.572374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:30.572374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:30.572374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:30.572374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:30.573297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:30.573297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:30.573297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:30.573297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:30.573297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:30.573297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:30.573297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:30.573297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:30.573297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:30.573297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:30.573297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:30.573297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:30.573297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:30.573297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:30.573297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:30.573297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:30.577885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:30.577885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:30.577885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:30.577885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:30.577885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:30.577885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:30.577885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:30.577885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:30.577885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:30.577885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:30.577885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:20:30.577885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:20:30.577885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 63/119] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:20:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:30.577885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb07df-86dc-4745-a91f-936795b7ff9c None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:30.618910 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07a74b-66c2-4b4c-be56-efba3d0a595d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:30.655351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce4de5-bb47-412d-9a33-43102dbef4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:30.656529 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce4de5-bb47-412d-9a33-43102dbef4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:30.657017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce4de5-bb47-412d-9a33-43102dbef4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:30.657532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce4de5-bb47-412d-9a33-43102dbef4a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:30.669271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce4de5-bb47-412d-9a33-43102dbef4a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:30.677841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-f4ce4de5-bb47-412d-9a33-43102dbef4a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:30.677841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:30.677841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:30.677841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:30.677841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:30.677841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:30.677841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:30.677841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:30.677841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:30.677841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:30.677841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:30.677841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:30.677841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:30.677841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:30.677841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:30.677841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:30.678721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:30.678721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:30.678721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:30.678721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:30.678721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:30.678721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:30.678721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:30.678721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:30.678721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:30.678721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:30.678721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:30.678721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:30.678721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:30.678721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:30.678721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:30.678721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:30.679512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:30.679512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:30.679512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:30.679512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:30.679512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:30.679512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:30.679512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:30.679512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:30.679512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:30.679512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:30.679512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:30.679512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:20:30.686850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 57/120] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:30.688477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8de97-ade9-40a6-9c05-0af61d505dcc None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:30.688973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8de97-ade9-40a6-9c05-0af61d505dcc None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:30.689488 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8de97-ade9-40a6-9c05-0af61d505dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:30.707408 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb07df-86dc-4745-a91f-936795b7ff9c None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:30.707734 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-f4f8de97-ade9-40a6-9c05-0af61d505dcc None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:30.709136 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f4f8de97-ade9-40a6-9c05-0af61d505dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:30.740612 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:30.751681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5f8cc-c14e-4060-b620-832fbc2ebbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:30.753518 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5f8cc-c14e-4060-b620-832fbc2ebbab None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:30.754034 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5f8cc-c14e-4060-b620-832fbc2ebbab None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:30.754513 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5f8cc-c14e-4060-b620-832fbc2ebbab None admin] PATH_INFO: `/v3/users/ed0016d7b075461eb857ad7527a36809` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:30.755969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5f8cc-c14e-4060-b620-832fbc2ebbab None admin] RBAC: Authorizing `identity:get_user(user_id=ed0016d7b075461eb857ad7527a36809)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:30.757779 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5f8cc-c14e-4060-b620-832fbc2ebbab None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:30.759896 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 64/121] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:30 2022] GET /identity/v3/users/ed0016d7b075461eb857ad7527a36809 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:30.761893 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5f8cc-c14e-4060-b620-832fbc2ebbab None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:30.805531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 58/122] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:30.805531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8de97-ade9-40a6-9c05-0af61d505dcc None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:30.864452 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:30.906857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5f8cc-c14e-4060-b620-832fbc2ebbab None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:30.912962 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8de97-ade9-40a6-9c05-0af61d505dcc None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:30.949400 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb74e5-de9b-482d-8a6c-787b16fa6e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:30.950232 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb74e5-de9b-482d-8a6c-787b16fa6e23 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:30.950375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ec61d-3dd0-4704-82b0-5b94de689a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:30.950879 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb74e5-de9b-482d-8a6c-787b16fa6e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:30.950981 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ec61d-3dd0-4704-82b0-5b94de689a6c None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:30.951277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ec61d-3dd0-4704-82b0-5b94de689a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:30.951566 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ec61d-3dd0-4704-82b0-5b94de689a6c None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:30.951845 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb74e5-de9b-482d-8a6c-787b16fa6e23 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:30.952329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ec61d-3dd0-4704-82b0-5b94de689a6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:30.952642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ec61d-3dd0-4704-82b0-5b94de689a6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:30.953301 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb74e5-de9b-482d-8a6c-787b16fa6e23 None admin] RBAC: Adding query filter params (name=member) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:30.953825 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb74e5-de9b-482d-8a6c-787b16fa6e23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:30.954056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ec61d-3dd0-4704-82b0-5b94de689a6c None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:30.955693 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb74e5-de9b-482d-8a6c-787b16fa6e23 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:30.963860 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 65/123] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:30.964514 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ec61d-3dd0-4704-82b0-5b94de689a6c None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:30.965419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 59/124] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:20:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:30.966540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb74e5-de9b-482d-8a6c-787b16fa6e23 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:31.035444 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:31.084159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb74e5-de9b-482d-8a6c-787b16fa6e23 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:31.090498 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ec61d-3dd0-4704-82b0-5b94de689a6c None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:31.103036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:31.124357 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9e8f3-652c-4c7f-82dd-965e0e7ff14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:31.125744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9e8f3-652c-4c7f-82dd-965e0e7ff14e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:31.126283 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9e8f3-652c-4c7f-82dd-965e0e7ff14e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:31.126913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9e8f3-652c-4c7f-82dd-965e0e7ff14e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:31.132193 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a44388-ef84-4a25-a823-b50a9791c612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:31.133107 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a44388-ef84-4a25-a823-b50a9791c612 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:31.133431 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a44388-ef84-4a25-a823-b50a9791c612 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:31.133721 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a44388-ef84-4a25-a823-b50a9791c612 None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:31.135330 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a44388-ef84-4a25-a823-b50a9791c612 None admin] RBAC: Authorizing `identity:get_project(project_id=8a3d0e211dd942aaa39eed1730d17e92)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:31.137194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a44388-ef84-4a25-a823-b50a9791c612 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:31.139243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 66/125] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:30 2022] GET /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:31.140449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a44388-ef84-4a25-a823-b50a9791c612 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:31.144797 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9e8f3-652c-4c7f-82dd-965e0e7ff14e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:31.154031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-bfa9e8f3-652c-4c7f-82dd-965e0e7ff14e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:20:31.154031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:31.154031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:31.154031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:31.154031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:31.154031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:31.154031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:31.154031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:31.154031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:31.154031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:31.154031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:31.154031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:31.154031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:31.154031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:31.154031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:31.154031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:31.154951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:31.154951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:31.154951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:31.154951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:31.154951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:31.154951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:31.154951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:31.154951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:31.154951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:31.154951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:31.154951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:31.154951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:31.154951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:31.154951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:31.154951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:31.154951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:31.155746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:31.155746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:31.155746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:31.155746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:31.155746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:31.155746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:31.155746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:31.155746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:31.155746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:31.155746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:31.155746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:20:31.155746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:20:31.158290 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 60/126] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:20:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:31.159892 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9e8f3-652c-4c7f-82dd-965e0e7ff14e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:31.266802 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a44388-ef84-4a25-a823-b50a9791c612 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:31.269932 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9e8f3-652c-4c7f-82dd-965e0e7ff14e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:31.295367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:31.306134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58d4c2-3795-4a08-9aae-2f0bc08f7411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:31.306556 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcca84b-b7b9-4384-9b46-65f9709b6937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:31.307047 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58d4c2-3795-4a08-9aae-2f0bc08f7411 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:31.307334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58d4c2-3795-4a08-9aae-2f0bc08f7411 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:31.307441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcca84b-b7b9-4384-9b46-65f9709b6937 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:31.307613 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58d4c2-3795-4a08-9aae-2f0bc08f7411 None admin] PATH_INFO: `/v3/users/eac0995e068547e3890aa6066bdecd84` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:31.307705 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcca84b-b7b9-4384-9b46-65f9709b6937 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:31.307993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcca84b-b7b9-4384-9b46-65f9709b6937 None admin] PATH_INFO: `/v3/users/ed0016d7b075461eb857ad7527a36809` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:31.308973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58d4c2-3795-4a08-9aae-2f0bc08f7411 None admin] RBAC: Authorizing `identity:get_user(user_id=eac0995e068547e3890aa6066bdecd84)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:31.309244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcca84b-b7b9-4384-9b46-65f9709b6937 None admin] RBAC: Authorizing `identity:get_user(user_id=ed0016d7b075461eb857ad7527a36809)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:31.310669 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58d4c2-3795-4a08-9aae-2f0bc08f7411 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:31.310784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcca84b-b7b9-4384-9b46-65f9709b6937 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:31.312495 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 67/127] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:31 2022] GET /identity/v3/users/ed0016d7b075461eb857ad7527a36809 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:31.312607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 61/128] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:31 2022] GET /identity/v3/users/eac0995e068547e3890aa6066bdecd84 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:31.313075 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcca84b-b7b9-4384-9b46-65f9709b6937 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:31.313277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58d4c2-3795-4a08-9aae-2f0bc08f7411 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:31.364234 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:31.419384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58d4c2-3795-4a08-9aae-2f0bc08f7411 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:31.431235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcca84b-b7b9-4384-9b46-65f9709b6937 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:31.461870 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5734000-54bd-4389-bdbd-19ebeea8705f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:31.462563 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5734000-54bd-4389-bdbd-19ebeea8705f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:31.462854 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5734000-54bd-4389-bdbd-19ebeea8705f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:31.463139 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5734000-54bd-4389-bdbd-19ebeea8705f None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:31.463990 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5734000-54bd-4389-bdbd-19ebeea8705f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:31.464208 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5734000-54bd-4389-bdbd-19ebeea8705f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:31.465629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5734000-54bd-4389-bdbd-19ebeea8705f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:31.479661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 62/129] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:20:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:31.479661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5734000-54bd-4389-bdbd-19ebeea8705f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:31.479931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908b133-b29b-48b3-b217-fa79bca9b120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:31.479931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908b133-b29b-48b3-b217-fa79bca9b120 None admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:31.479931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908b133-b29b-48b3-b217-fa79bca9b120 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:31.479931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908b133-b29b-48b3-b217-fa79bca9b120 None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92/users/ed0016d7b075461eb857ad7527a36809/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:31.481760 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908b133-b29b-48b3-b217-fa79bca9b120 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a3d0e211dd942aaa39eed1730d17e92, user_id=ed0016d7b075461eb857ad7527a36809, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:31.483237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908b133-b29b-48b3-b217-fa79bca9b120 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:31.504805 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 68/130] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:20:31 2022] PUT /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92/users/ed0016d7b075461eb857ad7527a36809/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:20:31.506327 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908b133-b29b-48b3-b217-fa79bca9b120 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:31.591904 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:31.603466 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5734000-54bd-4389-bdbd-19ebeea8705f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:31.624077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:31.652018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908b133-b29b-48b3-b217-fa79bca9b120 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:31.694849 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21cdbe-566b-4ed8-8297-f7184da2171b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:31.695760 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21cdbe-566b-4ed8-8297-f7184da2171b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:31.696057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21cdbe-566b-4ed8-8297-f7184da2171b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:31.696345 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21cdbe-566b-4ed8-8297-f7184da2171b None admin] PATH_INFO: `/v3/projects/ca04884049304c19bdf4ab8b556ac8db` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:31.697799 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21cdbe-566b-4ed8-8297-f7184da2171b None admin] RBAC: Authorizing `identity:get_project(project_id=ca04884049304c19bdf4ab8b556ac8db)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:31.699314 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21cdbe-566b-4ed8-8297-f7184da2171b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:31.701351 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 69/131] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:31 2022] GET /identity/v3/projects/ca04884049304c19bdf4ab8b556ac8db => generated 312 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:31.702225 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21cdbe-566b-4ed8-8297-f7184da2171b None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:31.711922 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f1cee-7bc5-47dd-9b14-93445fce96ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:31.712819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f1cee-7bc5-47dd-9b14-93445fce96ab None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:31.713110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f1cee-7bc5-47dd-9b14-93445fce96ab None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:31.713427 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f1cee-7bc5-47dd-9b14-93445fce96ab None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:31.714675 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f1cee-7bc5-47dd-9b14-93445fce96ab None admin] RBAC: Authorizing `identity:get_project(project_id=8a3d0e211dd942aaa39eed1730d17e92)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:31.716148 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f1cee-7bc5-47dd-9b14-93445fce96ab None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:31.718257 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 63/132] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:31 2022] GET /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:31.722506 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f1cee-7bc5-47dd-9b14-93445fce96ab None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:31.846226 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21cdbe-566b-4ed8-8297-f7184da2171b None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:31.852393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f1cee-7bc5-47dd-9b14-93445fce96ab None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:31.883878 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:31.913066 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe219b-e491-487c-b4d6-7913c494d6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:31.914010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe219b-e491-487c-b4d6-7913c494d6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:31.914300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe219b-e491-487c-b4d6-7913c494d6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:31.914580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe219b-e491-487c-b4d6-7913c494d6b1 None admin] PATH_INFO: `/v3/users/ed0016d7b075461eb857ad7527a36809` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:31.916283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe219b-e491-487c-b4d6-7913c494d6b1 None admin] RBAC: Authorizing `identity:get_user(user_id=ed0016d7b075461eb857ad7527a36809)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:31.917836 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe219b-e491-487c-b4d6-7913c494d6b1 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:31.921524 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 70/133] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:31 2022] GET /identity/v3/users/ed0016d7b075461eb857ad7527a36809 => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:31.923549 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa33d5-5406-4c99-8888-c334ce773776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:31.925109 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa33d5-5406-4c99-8888-c334ce773776 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:31.925438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa33d5-5406-4c99-8888-c334ce773776 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:31.925719 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa33d5-5406-4c99-8888-c334ce773776 None admin] PATH_INFO: `/v3/projects/ca04884049304c19bdf4ab8b556ac8db/users/ed0016d7b075461eb857ad7527a36809/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:31.927423 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe219b-e491-487c-b4d6-7913c494d6b1 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:31.929436 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa33d5-5406-4c99-8888-c334ce773776 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca04884049304c19bdf4ab8b556ac8db, user_id=ed0016d7b075461eb857ad7527a36809, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:31.930914 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa33d5-5406-4c99-8888-c334ce773776 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:31.967210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 64/134] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:20:31 2022] PUT /identity/v3/projects/ca04884049304c19bdf4ab8b556ac8db/users/ed0016d7b075461eb857ad7527a36809/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:20:31.967959 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa33d5-5406-4c99-8888-c334ce773776 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:32.055548 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:32.120163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe219b-e491-487c-b4d6-7913c494d6b1 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:32.152676 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa33d5-5406-4c99-8888-c334ce773776 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:32.170607 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b839b9-915e-48bd-8106-fff19239ded1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:32.171530 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b839b9-915e-48bd-8106-fff19239ded1 None admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:32.171821 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b839b9-915e-48bd-8106-fff19239ded1 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:32.172106 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b839b9-915e-48bd-8106-fff19239ded1 None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92/users/eac0995e068547e3890aa6066bdecd84/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:32.178702 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b839b9-915e-48bd-8106-fff19239ded1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a3d0e211dd942aaa39eed1730d17e92, user_id=eac0995e068547e3890aa6066bdecd84, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:32.180198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b839b9-915e-48bd-8106-fff19239ded1 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:32.189557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513b773-9445-4d30-b6cb-de7f159fa33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:32.191251 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513b773-9445-4d30-b6cb-de7f159fa33b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:32.191586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513b773-9445-4d30-b6cb-de7f159fa33b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:32.191944 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513b773-9445-4d30-b6cb-de7f159fa33b None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:32.193532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513b773-9445-4d30-b6cb-de7f159fa33b None admin] RBAC: Authorizing `identity:get_project(project_id=8a3d0e211dd942aaa39eed1730d17e92)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:32.195091 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513b773-9445-4d30-b6cb-de7f159fa33b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:32.196832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 65/135] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:31 2022] GET /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:32.198353 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba65f6-2eac-4d61-8942-68b86f1fc69b None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:32.198647 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba65f6-2eac-4d61-8942-68b86f1fc69b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:32.198929 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba65f6-2eac-4d61-8942-68b86f1fc69b None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:32.200435 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 66/136] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:32.202111 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 71/137] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:20:31 2022] PUT /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92/users/eac0995e068547e3890aa6066bdecd84/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:20:32.204897 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba65f6-2eac-4d61-8942-68b86f1fc69b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:32.206249 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a198fe8-daa0-4e44-96bb-53275aeaf7d1 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:32.206546 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a198fe8-daa0-4e44-96bb-53275aeaf7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:32.207109 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a198fe8-daa0-4e44-96bb-53275aeaf7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:32.250539 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-1a198fe8-daa0-4e44-96bb-53275aeaf7d1 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:32.251902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-1a198fe8-daa0-4e44-96bb-53275aeaf7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:32.325908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba65f6-2eac-4d61-8942-68b86f1fc69b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:32.329208 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:32.332005 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:32.358984 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 72/138] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:20:32.361275 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac21d26-975c-40d1-aaf5-5874b08303c4 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:32.362242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac21d26-975c-40d1-aaf5-5874b08303c4 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:32.363109 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac21d26-975c-40d1-aaf5-5874b08303c4 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:32.365268 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 73/139] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:32.365396 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6424359-5a4e-439a-937c-d189f42580fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:32.369199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6424359-5a4e-439a-937c-d189f42580fe None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:32.369199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6424359-5a4e-439a-937c-d189f42580fe None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:32.369199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6424359-5a4e-439a-937c-d189f42580fe None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92/users/ed0016d7b075461eb857ad7527a36809/roles/1423a24997e840608b6c522958481fde` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:32.371250 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e1f69-d923-47a1-9407-6941e59c2fde None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:32.371551 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e1f69-d923-47a1-9407-6941e59c2fde None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:32.371841 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e1f69-d923-47a1-9407-6941e59c2fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:32.372188 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6424359-5a4e-439a-937c-d189f42580fe None admin] RBAC: Authorizing `identity:create_grant(project_id=8a3d0e211dd942aaa39eed1730d17e92, user_id=ed0016d7b075461eb857ad7527a36809, role_id=1423a24997e840608b6c522958481fde)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:32.374102 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6424359-5a4e-439a-937c-d189f42580fe None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:32.390008 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 67/140] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:20:32 2022] PUT /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92/users/ed0016d7b075461eb857ad7527a36809/roles/1423a24997e840608b6c522958481fde => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:20:32.391328 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-6d4e1f69-d923-47a1-9407-6941e59c2fde None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:32.392576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-6d4e1f69-d923-47a1-9407-6941e59c2fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:32.397326 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6424359-5a4e-439a-937c-d189f42580fe None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:32.543409 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 74/141] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:20:32.602359 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e1f69-d923-47a1-9407-6941e59c2fde None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:32.619290 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6424359-5a4e-439a-937c-d189f42580fe None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:32.692020 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ba9e5-5631-46e7-9090-950fef5d6acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:32.692843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ba9e5-5631-46e7-9090-950fef5d6acf None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:32.693139 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ba9e5-5631-46e7-9090-950fef5d6acf None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:32.693567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ba9e5-5631-46e7-9090-950fef5d6acf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:32.697059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ba9e5-5631-46e7-9090-950fef5d6acf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:32.698574 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ba9e5-5631-46e7-9090-950fef5d6acf None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:32.700492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 68/142] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:20:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:32.706739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ba9e5-5631-46e7-9090-950fef5d6acf None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:32.744708 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e1f69-d923-47a1-9407-6941e59c2fde None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:32.757803 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:32.774591 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:32.789690 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ac606-4e81-4d57-b9cf-e0a7dda7b030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:32.790516 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ac606-4e81-4d57-b9cf-e0a7dda7b030 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:32.790816 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ac606-4e81-4d57-b9cf-e0a7dda7b030 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:32.791117 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ac606-4e81-4d57-b9cf-e0a7dda7b030 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:32.808213 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ac606-4e81-4d57-b9cf-e0a7dda7b030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:32.819430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-1d2ac606-4e81-4d57-b9cf-e0a7dda7b030 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:20:32.819430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:32.819430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:32.819430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:32.819430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:32.819430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:32.819430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:32.819430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:32.819430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:32.819430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:32.819430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:32.819430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:32.819430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:32.819430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:32.819430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:32.819430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:32.820403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:32.820403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:32.820403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:32.820403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:32.820403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:32.820403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:32.820403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:32.820403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:32.820403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:32.820403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:32.820403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:32.820403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:32.820403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:32.820403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:32.820403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:32.820403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:32.821240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:32.821240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:32.821240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:32.821240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:32.821240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:32.821240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:32.821240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:32.821240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:32.821240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:32.821240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:32.821240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:20:32.821240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:20:32.822357 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 75/143] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:32.832895 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ac606-4e81-4d57-b9cf-e0a7dda7b030 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:32.963129 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ba9e5-5631-46e7-9090-950fef5d6acf None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:33.029970 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ac606-4e81-4d57-b9cf-e0a7dda7b030 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:33.058746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce4448-5067-49ab-9703-88f81a926e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:33.059680 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce4448-5067-49ab-9703-88f81a926e4d None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:33.060128 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce4448-5067-49ab-9703-88f81a926e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:33.060556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce4448-5067-49ab-9703-88f81a926e4d None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:33.061479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ce4448-5067-49ab-9703-88f81a926e4d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:33.063160 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ce4448-5067-49ab-9703-88f81a926e4d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:33.109821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 69/144] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 17:20:32 2022] POST /identity/v3/users => generated 283 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:20:33.114085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c3429-59aa-4694-8cbf-e913ae272d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:33.115013 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c3429-59aa-4694-8cbf-e913ae272d2f None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:33.115451 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c3429-59aa-4694-8cbf-e913ae272d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:33.115907 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c3429-59aa-4694-8cbf-e913ae272d2f None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:33.116865 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c3429-59aa-4694-8cbf-e913ae272d2f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:33.117372 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c3429-59aa-4694-8cbf-e913ae272d2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:33.118982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c3429-59aa-4694-8cbf-e913ae272d2f None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:33.135352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 76/145] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:20:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:33.142277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce4448-5067-49ab-9703-88f81a926e4d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:33.272004 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:33.291086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce4448-5067-49ab-9703-88f81a926e4d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:33.341968 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926f336-5ec9-475e-96ac-15153a526062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:33.342802 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926f336-5ec9-475e-96ac-15153a526062 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:33.343093 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926f336-5ec9-475e-96ac-15153a526062 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:33.343379 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926f336-5ec9-475e-96ac-15153a526062 None admin] PATH_INFO: `/v3/users/a22703a1b8784cb888c43670d59fb559` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:33.346090 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926f336-5ec9-475e-96ac-15153a526062 None admin] RBAC: Authorizing `identity:get_user(user_id=a22703a1b8784cb888c43670d59fb559)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:33.347855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926f336-5ec9-475e-96ac-15153a526062 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:33.349867 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 70/146] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:33 2022] GET /identity/v3/users/a22703a1b8784cb888c43670d59fb559 => generated 289 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:33.358252 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c3429-59aa-4694-8cbf-e913ae272d2f None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:33.430520 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:33.532684 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c3429-59aa-4694-8cbf-e913ae272d2f None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:33.585502 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdae5533-d51c-4e68-bd2a-e1f68d8b5051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:33.586384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae5533-d51c-4e68-bd2a-e1f68d8b5051 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:33.586678 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae5533-d51c-4e68-bd2a-e1f68d8b5051 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:33.586966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae5533-d51c-4e68-bd2a-e1f68d8b5051 None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:33.588193 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdae5533-d51c-4e68-bd2a-e1f68d8b5051 None admin] RBAC: Authorizing `identity:get_project(project_id=8a3d0e211dd942aaa39eed1730d17e92)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:33.589706 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdae5533-d51c-4e68-bd2a-e1f68d8b5051 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:33.591776 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 77/147] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:33 2022] GET /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:33.598297 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926f336-5ec9-475e-96ac-15153a526062 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:33.739218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926f336-5ec9-475e-96ac-15153a526062 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:33.783812 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe2890-08ba-4315-9fac-9b747577ac54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:33.784691 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe2890-08ba-4315-9fac-9b747577ac54 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:33.784979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe2890-08ba-4315-9fac-9b747577ac54 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:33.785284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe2890-08ba-4315-9fac-9b747577ac54 None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92/users/a22703a1b8784cb888c43670d59fb559/roles/075fc2a9c4184b9db26b044f3d963233` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:33.787356 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe2890-08ba-4315-9fac-9b747577ac54 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a3d0e211dd942aaa39eed1730d17e92, user_id=a22703a1b8784cb888c43670d59fb559, role_id=075fc2a9c4184b9db26b044f3d963233)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:33.788823 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe2890-08ba-4315-9fac-9b747577ac54 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:33.818376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 71/148] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:20:33 2022] PUT /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92/users/a22703a1b8784cb888c43670d59fb559/roles/075fc2a9c4184b9db26b044f3d963233 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:20:34.688575 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5707f8-5b82-4532-b8b7-0bb75cec04f8 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:34.689080 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5707f8-5b82-4532-b8b7-0bb75cec04f8 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:34.689172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5707f8-5b82-4532-b8b7-0bb75cec04f8 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:34.691927 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 78/149] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:34.697374 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3cdf5-70ab-4bda-9560-344da83fee16 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:34.697835 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3cdf5-70ab-4bda-9560-344da83fee16 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:34.698006 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3cdf5-70ab-4bda-9560-344da83fee16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:34.722259 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-eec3cdf5-70ab-4bda-9560-344da83fee16 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:34.723504 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-eec3cdf5-70ab-4bda-9560-344da83fee16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:34.736280 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:34.809244 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 72/150] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:34.848428 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5707f8-5b82-4532-b8b7-0bb75cec04f8 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:34.979378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:34.985899 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5707f8-5b82-4532-b8b7-0bb75cec04f8 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:35.025317 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706b03b-50e9-4655-9ecf-078b64850e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:35.026192 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706b03b-50e9-4655-9ecf-078b64850e73 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:35.026486 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706b03b-50e9-4655-9ecf-078b64850e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:35.026766 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706b03b-50e9-4655-9ecf-078b64850e73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:35.038305 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2706b03b-50e9-4655-9ecf-078b64850e73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:35.046350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-2706b03b-50e9-4655-9ecf-078b64850e73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:35.046350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:35.046350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:35.046350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:35.046350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:35.046350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:35.046350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:35.046350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:35.046350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:35.046350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:35.046350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:35.046350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:35.046350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:35.046350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:35.046350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:35.046350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:35.047218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:35.047218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:35.047218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:35.047218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:35.047218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:35.047218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:35.047218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:35.047218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:35.047218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:35.047218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:35.047218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:35.047218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:35.047218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:35.047218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:35.047218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:35.047218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:35.048232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:35.048232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:35.048232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:35.048232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:35.048232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:35.048232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:35.048232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:35.048232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:35.048232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:35.048232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:35.048232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:35.048232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:20:35.048232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 79/151] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:35.054347 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3cdf5-70ab-4bda-9560-344da83fee16 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:35.216350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3cdf5-70ab-4bda-9560-344da83fee16 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:35.280233 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fce55-ea38-4952-b2dc-43a30de75dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:35.280928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fce55-ea38-4952-b2dc-43a30de75dca None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:35.281244 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fce55-ea38-4952-b2dc-43a30de75dca None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:35.281530 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fce55-ea38-4952-b2dc-43a30de75dca None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:35.282297 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03fce55-ea38-4952-b2dc-43a30de75dca None admin] RBAC: Adding query filter params (name=member) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:35.282613 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03fce55-ea38-4952-b2dc-43a30de75dca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:35.284595 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03fce55-ea38-4952-b2dc-43a30de75dca None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:35.296866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 73/152] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:20:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:35.303218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706b03b-50e9-4655-9ecf-078b64850e73 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:35.436343 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706b03b-50e9-4655-9ecf-078b64850e73 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:35.474961 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1889ff7e-12b3-4a65-a272-fac597865f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:35.475853 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1889ff7e-12b3-4a65-a272-fac597865f60 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:35.476142 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1889ff7e-12b3-4a65-a272-fac597865f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:35.476426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1889ff7e-12b3-4a65-a272-fac597865f60 None admin] PATH_INFO: `/v3/users/ed0016d7b075461eb857ad7527a36809` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:35.477674 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1889ff7e-12b3-4a65-a272-fac597865f60 None admin] RBAC: Authorizing `identity:get_user(user_id=ed0016d7b075461eb857ad7527a36809)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:35.479151 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1889ff7e-12b3-4a65-a272-fac597865f60 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:35.481018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 80/153] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:35 2022] GET /identity/v3/users/ed0016d7b075461eb857ad7527a36809 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:35.486473 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fce55-ea38-4952-b2dc-43a30de75dca None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:35.499910 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:35.607193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fce55-ea38-4952-b2dc-43a30de75dca None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:35.652389 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70960e45-5b4a-4c8b-a2c6-aa7cd6f69029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:35.653059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70960e45-5b4a-4c8b-a2c6-aa7cd6f69029 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:35.653371 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70960e45-5b4a-4c8b-a2c6-aa7cd6f69029 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:35.653654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70960e45-5b4a-4c8b-a2c6-aa7cd6f69029 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:35.663316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114175a-11e8-41b8-9ca9-a85f51421ccb None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:35.663629 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114175a-11e8-41b8-9ca9-a85f51421ccb None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:35.663956 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114175a-11e8-41b8-9ca9-a85f51421ccb None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:35.665659 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 81/154] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:20:35.671015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb1ad0-4541-4076-a93d-fb834f3eb77e None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:35.671309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb1ad0-4541-4076-a93d-fb834f3eb77e None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:35.671594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb1ad0-4541-4076-a93d-fb834f3eb77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:35.706806 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-44bb1ad0-4541-4076-a93d-fb834f3eb77e None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:35.709586 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-44bb1ad0-4541-4076-a93d-fb834f3eb77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:35.764990 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:35.843628 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70960e45-5b4a-4c8b-a2c6-aa7cd6f69029 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:35.845121 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70960e45-5b4a-4c8b-a2c6-aa7cd6f69029 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:35.869099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:35.878628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 82/155] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:20:35.887255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1077ced-f4d8-40a6-8c5f-ff752acb2fd9 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:35.888016 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1077ced-f4d8-40a6-8c5f-ff752acb2fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:35.888496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1077ced-f4d8-40a6-8c5f-ff752acb2fd9 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:35.890994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 83/156] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:35 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:35.896917 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a9b7d-5853-48d7-8419-ea1405f16ded None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:35.897525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a9b7d-5853-48d7-8419-ea1405f16ded None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:35.898032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a9b7d-5853-48d7-8419-ea1405f16ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:35.935448 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-aa0a9b7d-5853-48d7-8419-ea1405f16ded None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:35.937115 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-aa0a9b7d-5853-48d7-8419-ea1405f16ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:36.023468 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 74/157] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:20:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:20:36.024697 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70960e45-5b4a-4c8b-a2c6-aa7cd6f69029 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:36.054088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 84/158] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:36.055500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb0636-cf19-4b0f-b11c-0ed6ab83af14 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:36.055872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb0636-cf19-4b0f-b11c-0ed6ab83af14 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:36.056126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb0636-cf19-4b0f-b11c-0ed6ab83af14 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:36.057380 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 85/159] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:36.058073 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb0636-cf19-4b0f-b11c-0ed6ab83af14 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:36.136349 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70960e45-5b4a-4c8b-a2c6-aa7cd6f69029 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:36.147355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:36.178325 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e7011-0305-4fb1-9232-13b9f696f358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:36.179650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e7011-0305-4fb1-9232-13b9f696f358 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:36.180964 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e7011-0305-4fb1-9232-13b9f696f358 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:36.182215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e7011-0305-4fb1-9232-13b9f696f358 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:36.202955 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e7011-0305-4fb1-9232-13b9f696f358 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:36.213271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-017e7011-0305-4fb1-9232-13b9f696f358 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:36.213271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:36.213271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:36.213271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:36.213271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:36.213271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:36.213271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:36.213271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:36.213271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:36.213271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:36.213271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:36.213271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:36.213271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:36.213271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:36.213271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:36.213271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:36.214284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:36.214284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:36.214284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:36.214284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:36.214284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:36.214284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:36.214284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:36.214284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:36.214284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:36.214284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:36.214284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:36.214284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:36.214284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:36.214284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:36.214284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:36.214284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:36.215074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:36.215074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:36.215074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:36.215074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:36.215074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:36.215074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:36.215074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:36.215074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:36.215074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:36.215074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:36.215074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:36.215074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:20:36.217965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 75/160] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:36.219467 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb04da-4d7f-4d7a-8266-6ee314adc215 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:36.219955 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb04da-4d7f-4d7a-8266-6ee314adc215 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:36.220422 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb04da-4d7f-4d7a-8266-6ee314adc215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:36.235201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:36.249168 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb0636-cf19-4b0f-b11c-0ed6ab83af14 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:36.271663 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-9eeb04da-4d7f-4d7a-8266-6ee314adc215 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:36.273312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9eeb04da-4d7f-4d7a-8266-6ee314adc215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:36.287656 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe2b75-d89b-4e3a-9407-c82c044692fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:36.288924 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe2b75-d89b-4e3a-9407-c82c044692fd None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:36.289484 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe2b75-d89b-4e3a-9407-c82c044692fd None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:36.289969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe2b75-d89b-4e3a-9407-c82c044692fd None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:36.291379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe2b75-d89b-4e3a-9407-c82c044692fd None admin] RBAC: Authorizing `identity:get_project(project_id=8a3d0e211dd942aaa39eed1730d17e92)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:36.293060 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe2b75-d89b-4e3a-9407-c82c044692fd None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:36.295130 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 86/161] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:36 2022] GET /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:36.296246 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe2b75-d89b-4e3a-9407-c82c044692fd None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:36.387943 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 76/162] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:36.388942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb04da-4d7f-4d7a-8266-6ee314adc215 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:36.414743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe2b75-d89b-4e3a-9407-c82c044692fd None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:36.467045 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5f25c-726e-4f96-9fe1-dafd1dc33651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:36.468257 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5f25c-726e-4f96-9fe1-dafd1dc33651 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:36.468751 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5f25c-726e-4f96-9fe1-dafd1dc33651 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:36.469274 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5f25c-726e-4f96-9fe1-dafd1dc33651 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:36.481351 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b5f25c-726e-4f96-9fe1-dafd1dc33651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:36.491859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-f3b5f25c-726e-4f96-9fe1-dafd1dc33651 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:20:36.491859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:36.491859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:36.491859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:36.491859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:36.491859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:36.491859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:36.491859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:36.491859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:36.491859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:36.491859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:36.491859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:36.491859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:36.491859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:36.491859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:36.491859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:36.492797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:36.492797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:36.492797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:36.492797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:36.492797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:36.492797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:36.492797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:36.492797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:36.492797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:36.492797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:36.492797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:36.492797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:36.492797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:36.492797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:36.492797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:36.492797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:36.493711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:36.493711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:36.493711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:36.493711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:36.493711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:36.493711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:36.493711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:36.493711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:36.493711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:36.493711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:36.493711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:20:36.493711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:20:36.496125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 87/163] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:20:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:36.497133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5f25c-726e-4f96-9fe1-dafd1dc33651 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:36.522944 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:36.543628 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:36.548954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb04da-4d7f-4d7a-8266-6ee314adc215 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:36.608780 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc3e36-5cf0-4e55-a94d-77dd32598fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:36.609508 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc3e36-5cf0-4e55-a94d-77dd32598fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:36.610030 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc3e36-5cf0-4e55-a94d-77dd32598fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:36.610138 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc3e36-5cf0-4e55-a94d-77dd32598fd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:36.610888 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc3e36-5cf0-4e55-a94d-77dd32598fd1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:36.611209 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc3e36-5cf0-4e55-a94d-77dd32598fd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:36.613191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc3e36-5cf0-4e55-a94d-77dd32598fd1 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:36.620994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5f25c-726e-4f96-9fe1-dafd1dc33651 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:36.622046 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 77/164] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:20:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:36.623075 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc3e36-5cf0-4e55-a94d-77dd32598fd1 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:36.661005 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b547751-be9f-4720-ac8d-da8052a3be6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:36.662168 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b547751-be9f-4720-ac8d-da8052a3be6e None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:36.662664 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b547751-be9f-4720-ac8d-da8052a3be6e None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:36.663146 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b547751-be9f-4720-ac8d-da8052a3be6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:36.664230 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b547751-be9f-4720-ac8d-da8052a3be6e None admin] RBAC: Adding query filter params (role.id=483146fdc4ac47cf97c04a9f66c091b4, scope.project.id=8a3d0e211dd942aaa39eed1730d17e92, user.id=ed0016d7b075461eb857ad7527a36809) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:36.664827 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b547751-be9f-4720-ac8d-da8052a3be6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:36.666567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b547751-be9f-4720-ac8d-da8052a3be6e None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:36.673133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 88/165] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:20:36 2022] GET /identity/v3/role_assignments?user.id=ed0016d7b075461eb857ad7527a36809&scope.project.id=8a3d0e211dd942aaa39eed1730d17e92&role.id=483146fdc4ac47cf97c04a9f66c091b4 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:36.674380 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b547751-be9f-4720-ac8d-da8052a3be6e None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:36.794539 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:36.816749 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc3e36-5cf0-4e55-a94d-77dd32598fd1 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:36.830378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b547751-be9f-4720-ac8d-da8052a3be6e None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:36.898639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc068e5-5fb3-480f-8d51-0ef0199cb141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:36.899391 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc068e5-5fb3-480f-8d51-0ef0199cb141 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:36.899688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc068e5-5fb3-480f-8d51-0ef0199cb141 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:36.899977 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc068e5-5fb3-480f-8d51-0ef0199cb141 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:36.900800 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc068e5-5fb3-480f-8d51-0ef0199cb141 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:36.901123 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc068e5-5fb3-480f-8d51-0ef0199cb141 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:36.902625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc068e5-5fb3-480f-8d51-0ef0199cb141 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:36.906021 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f10510-8a13-4c6c-915c-a96bd1b20bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:36.906871 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f10510-8a13-4c6c-915c-a96bd1b20bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:36.907159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f10510-8a13-4c6c-915c-a96bd1b20bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:36.907437 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f10510-8a13-4c6c-915c-a96bd1b20bb2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:36.921452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 89/166] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:20:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:36.922358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc068e5-5fb3-480f-8d51-0ef0199cb141 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:36.930266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f10510-8a13-4c6c-915c-a96bd1b20bb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:36.954472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-85f10510-8a13-4c6c-915c-a96bd1b20bb2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:20:36.954472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:36.954472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:36.954472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:36.954472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:36.954472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:36.954472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:36.954472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:36.954472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:36.954472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:36.954472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:36.954472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:36.954472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:36.954472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:36.954472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:36.954472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:36.955484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:36.955484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:36.955484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:36.955484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:36.955484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:36.955484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:36.955484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:36.955484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:36.955484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:36.955484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:36.955484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:36.955484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:36.955484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:36.955484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:36.955484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:36.955484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:36.956773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:36.956773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:36.956773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:36.956773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:36.956773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:36.956773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:36.956773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:36.956773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:36.956773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:36.956773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:36.956773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:20:36.956773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:20:36.956773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 78/167] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:20:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:36.957751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f10510-8a13-4c6c-915c-a96bd1b20bb2 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:36.997588 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:37.095295 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc068e5-5fb3-480f-8d51-0ef0199cb141 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:37.116165 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f10510-8a13-4c6c-915c-a96bd1b20bb2 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:37.149297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f20f5-d507-4614-a72a-a06f71b2c34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:37.150259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f20f5-d507-4614-a72a-a06f71b2c34a None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:37.150554 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f20f5-d507-4614-a72a-a06f71b2c34a None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:37.150857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f20f5-d507-4614-a72a-a06f71b2c34a None admin] PATH_INFO: `/v3/users/ed0016d7b075461eb857ad7527a36809` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:37.152136 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f20f5-d507-4614-a72a-a06f71b2c34a None admin] RBAC: Authorizing `identity:get_user(user_id=ed0016d7b075461eb857ad7527a36809)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:37.154481 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f20f5-d507-4614-a72a-a06f71b2c34a None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:37.156173 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 90/168] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:36 2022] GET /identity/v3/users/ed0016d7b075461eb857ad7527a36809 => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:37.156951 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f20f5-d507-4614-a72a-a06f71b2c34a None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:37.167241 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab72aba-bb5d-4d2a-8b82-70bef6a0d582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:37.168149 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab72aba-bb5d-4d2a-8b82-70bef6a0d582 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:37.168435 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab72aba-bb5d-4d2a-8b82-70bef6a0d582 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:37.168716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab72aba-bb5d-4d2a-8b82-70bef6a0d582 None admin] PATH_INFO: `/v3/users/ed0016d7b075461eb857ad7527a36809` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:37.169978 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab72aba-bb5d-4d2a-8b82-70bef6a0d582 None admin] RBAC: Authorizing `identity:get_user(user_id=ed0016d7b075461eb857ad7527a36809)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:37.171453 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab72aba-bb5d-4d2a-8b82-70bef6a0d582 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:37.177997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 79/169] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:36 2022] GET /identity/v3/users/ed0016d7b075461eb857ad7527a36809 => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:37.178615 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:37.178914 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab72aba-bb5d-4d2a-8b82-70bef6a0d582 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:37.280610 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f20f5-d507-4614-a72a-a06f71b2c34a None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:37.324534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d09614-e11c-448f-adf3-667da6200bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:37.325259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d09614-e11c-448f-adf3-667da6200bfe None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:37.325556 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d09614-e11c-448f-adf3-667da6200bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:37.325843 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d09614-e11c-448f-adf3-667da6200bfe None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:37.329475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d09614-e11c-448f-adf3-667da6200bfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:37.329805 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d09614-e11c-448f-adf3-667da6200bfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:37.331708 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d09614-e11c-448f-adf3-667da6200bfe None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:37.352054 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 91/170] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:37.352839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d09614-e11c-448f-adf3-667da6200bfe None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:37.375366 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:37.393932 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab72aba-bb5d-4d2a-8b82-70bef6a0d582 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:37.441689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec30c9a-8557-44d7-86a2-7dc885130f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:37.442374 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec30c9a-8557-44d7-86a2-7dc885130f16 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:37.442668 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec30c9a-8557-44d7-86a2-7dc885130f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:37.442959 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec30c9a-8557-44d7-86a2-7dc885130f16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:37.505689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:37.547414 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d09614-e11c-448f-adf3-667da6200bfe None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:37.584739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec30c9a-8557-44d7-86a2-7dc885130f16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:37.590356 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec30c9a-8557-44d7-86a2-7dc885130f16 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:37.609939 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da71976-6368-41f1-8214-4eb7c7affb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:37.610546 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da71976-6368-41f1-8214-4eb7c7affb88 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:37.610854 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da71976-6368-41f1-8214-4eb7c7affb88 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:37.611135 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da71976-6368-41f1-8214-4eb7c7affb88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:37.701047 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:37.719889 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 80/171] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:20:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:20:37.720756 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec30c9a-8557-44d7-86a2-7dc885130f16 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:37.737894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da71976-6368-41f1-8214-4eb7c7affb88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:37.739495 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da71976-6368-41f1-8214-4eb7c7affb88 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:37.823442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec30c9a-8557-44d7-86a2-7dc885130f16 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:37.870278 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9cc356-69fd-4e30-be20-a410edb4c353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:37.871083 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cc356-69fd-4e30-be20-a410edb4c353 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:37.871369 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cc356-69fd-4e30-be20-a410edb4c353 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:37.871645 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cc356-69fd-4e30-be20-a410edb4c353 None admin] PATH_INFO: `/v3/users/eac0995e068547e3890aa6066bdecd84` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:37.878075 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9cc356-69fd-4e30-be20-a410edb4c353 None admin] RBAC: Authorizing `identity:get_user(user_id=eac0995e068547e3890aa6066bdecd84)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:37.879762 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9cc356-69fd-4e30-be20-a410edb4c353 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:37.881537 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 81/172] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:37 2022] GET /identity/v3/users/eac0995e068547e3890aa6066bdecd84 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:37.883338 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:37.891668 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9cc356-69fd-4e30-be20-a410edb4c353 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:37.932398 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 92/173] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:20:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:20:37.933788 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864121a3-527a-4ef6-b073-5ad782d17a6d None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:37.934083 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864121a3-527a-4ef6-b073-5ad782d17a6d None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:37.934366 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864121a3-527a-4ef6-b073-5ad782d17a6d None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:37.936043 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 93/174] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:37.936768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864121a3-527a-4ef6-b073-5ad782d17a6d None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:37.972928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:38.041968 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9cc356-69fd-4e30-be20-a410edb4c353 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:38.081084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45066b4-0207-44e3-8454-c33662042341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:38.082131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45066b4-0207-44e3-8454-c33662042341 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:38.082419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45066b4-0207-44e3-8454-c33662042341 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:38.082697 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45066b4-0207-44e3-8454-c33662042341 None admin] PATH_INFO: `/v3/projects/ca04884049304c19bdf4ab8b556ac8db` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:38.083983 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45066b4-0207-44e3-8454-c33662042341 None admin] RBAC: Authorizing `identity:get_project(project_id=ca04884049304c19bdf4ab8b556ac8db)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:38.085549 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45066b4-0207-44e3-8454-c33662042341 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:38.087279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 82/175] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:37 2022] GET /identity/v3/projects/ca04884049304c19bdf4ab8b556ac8db => generated 312 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:38.088122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45066b4-0207-44e3-8454-c33662042341 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:38.142021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864121a3-527a-4ef6-b073-5ad782d17a6d None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:38.201930 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45066b4-0207-44e3-8454-c33662042341 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:38.222745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1a788-0e36-421c-b361-12c7e0dc2ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:38.223423 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1a788-0e36-421c-b361-12c7e0dc2ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:38.223714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1a788-0e36-421c-b361-12c7e0dc2ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:38.223999 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1a788-0e36-421c-b361-12c7e0dc2ee9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:38.237284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:38.253770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea1cad2-3cd6-4f77-a060-042a5918d30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:38.254924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1cad2-3cd6-4f77-a060-042a5918d30e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:38.255203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1cad2-3cd6-4f77-a060-042a5918d30e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:38.255477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1cad2-3cd6-4f77-a060-042a5918d30e None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:38.256800 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea1cad2-3cd6-4f77-a060-042a5918d30e None admin] RBAC: Authorizing `identity:get_project(project_id=8a3d0e211dd942aaa39eed1730d17e92)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:38.258549 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea1cad2-3cd6-4f77-a060-042a5918d30e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:38.261103 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 83/176] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:38 2022] GET /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:38.262508 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443413d9-9765-45f5-950b-60813a6a7807 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:38.262807 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443413d9-9765-45f5-950b-60813a6a7807 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:38.263084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443413d9-9765-45f5-950b-60813a6a7807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:38.286298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-443413d9-9765-45f5-950b-60813a6a7807 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:38.287515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-443413d9-9765-45f5-950b-60813a6a7807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:38.296869 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:38.334336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1a788-0e36-421c-b361-12c7e0dc2ee9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:38.336562 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1a788-0e36-421c-b361-12c7e0dc2ee9 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:38.361318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 84/177] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:38.361318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443413d9-9765-45f5-950b-60813a6a7807 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:38.445696 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 94/178] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:20:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:20:38.451596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea7fa7-fab0-48a9-b474-cb2dc4367bcd None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:38.451944 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea7fa7-fab0-48a9-b474-cb2dc4367bcd None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:38.452178 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea7fa7-fab0-48a9-b474-cb2dc4367bcd None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:38.454772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 95/179] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:38.459344 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea7fa7-fab0-48a9-b474-cb2dc4367bcd None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:38.474075 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443413d9-9765-45f5-950b-60813a6a7807 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:38.499839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:38.513491 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c190ab-fb4c-4251-ba55-f159761a3510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:38.514170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c190ab-fb4c-4251-ba55-f159761a3510 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:38.514460 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c190ab-fb4c-4251-ba55-f159761a3510 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:38.514745 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c190ab-fb4c-4251-ba55-f159761a3510 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:38.515584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c190ab-fb4c-4251-ba55-f159761a3510 None admin] RBAC: Adding query filter params (role.id=483146fdc4ac47cf97c04a9f66c091b4, scope.project.id=ca04884049304c19bdf4ab8b556ac8db, user.id=ed0016d7b075461eb857ad7527a36809) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:38.515947 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c190ab-fb4c-4251-ba55-f159761a3510 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:38.517383 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c190ab-fb4c-4251-ba55-f159761a3510 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:38.523727 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 85/180] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:20:38 2022] GET /identity/v3/role_assignments?user.id=ed0016d7b075461eb857ad7527a36809&scope.project.id=ca04884049304c19bdf4ab8b556ac8db&role.id=483146fdc4ac47cf97c04a9f66c091b4 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:38.524939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca07b7e-8fcd-4d57-9073-6faa4d730e2b None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:38.525705 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca07b7e-8fcd-4d57-9073-6faa4d730e2b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:38.525990 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca07b7e-8fcd-4d57-9073-6faa4d730e2b None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:38.529355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 86/181] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:38.529355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca07b7e-8fcd-4d57-9073-6faa4d730e2b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:38.547318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:38.582235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea7fa7-fab0-48a9-b474-cb2dc4367bcd None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:38.624624 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92601e25-029f-4a3d-ab94-4ed0e9aa41a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:38.625722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92601e25-029f-4a3d-ab94-4ed0e9aa41a8 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:38.626011 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92601e25-029f-4a3d-ab94-4ed0e9aa41a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:38.626288 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92601e25-029f-4a3d-ab94-4ed0e9aa41a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:38.628165 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92601e25-029f-4a3d-ab94-4ed0e9aa41a8 None admin] RBAC: Adding query filter params (role.id=1423a24997e840608b6c522958481fde, scope.project.id=8a3d0e211dd942aaa39eed1730d17e92, user.id=ed0016d7b075461eb857ad7527a36809) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:38.628424 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92601e25-029f-4a3d-ab94-4ed0e9aa41a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:38.629996 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92601e25-029f-4a3d-ab94-4ed0e9aa41a8 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:38.645194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 96/182] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:20:38 2022] GET /identity/v3/role_assignments?user.id=ed0016d7b075461eb857ad7527a36809&scope.project.id=8a3d0e211dd942aaa39eed1730d17e92&role.id=1423a24997e840608b6c522958481fde => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:38.645194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92601e25-029f-4a3d-ab94-4ed0e9aa41a8 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:38.653107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca07b7e-8fcd-4d57-9073-6faa4d730e2b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:38.695678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d132bb8-03a2-491c-9abb-fbc1e99a7fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:38.696637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d132bb8-03a2-491c-9abb-fbc1e99a7fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:38.696854 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d132bb8-03a2-491c-9abb-fbc1e99a7fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:38.697214 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d132bb8-03a2-491c-9abb-fbc1e99a7fc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:38.711658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d132bb8-03a2-491c-9abb-fbc1e99a7fc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:38.720474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-7d132bb8-03a2-491c-9abb-fbc1e99a7fc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:20:38.720474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:38.720474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:38.720474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:38.720474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:38.720474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:38.720474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:38.720474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:38.720474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:38.720474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:38.720474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:38.720474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:38.720474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:38.720474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:38.720474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:38.720474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:38.722350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:38.722350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:38.722350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:38.722350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:38.722350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:38.722350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:38.722350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:38.722350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:38.722350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:38.722350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:38.722350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:38.722350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:38.722350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:38.722350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:38.722350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:38.722350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:38.723170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:38.723170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:38.723170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:38.723170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:38.723170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:38.723170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:38.723170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:38.723170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:38.723170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:38.723170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:38.723170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:20:38.723170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:20:38.723170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 87/183] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:20:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:38.723855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8553187-868b-43a1-baca-8592ded61a64 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:38.723855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8553187-868b-43a1-baca-8592ded61a64 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:38.723855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8553187-868b-43a1-baca-8592ded61a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:38.750806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-c8553187-868b-43a1-baca-8592ded61a64 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:38.752050 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c8553187-868b-43a1-baca-8592ded61a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:38.802892 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92601e25-029f-4a3d-ab94-4ed0e9aa41a8 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:38.817112 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:38.846062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 88/184] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:38.847489 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6d58c-8390-4cd1-95f4-ca1d622d47d8 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:38.847905 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6d58c-8390-4cd1-95f4-ca1d622d47d8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:38.848194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6d58c-8390-4cd1-95f4-ca1d622d47d8 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:38.849646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 89/185] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:38.850903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10003e98-ca68-48d3-857f-7e5bcc0ec097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:38.851026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddfdab-97ed-42f9-8bb6-319f1698a233 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:38.851117 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10003e98-ca68-48d3-857f-7e5bcc0ec097 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:38.851189 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddfdab-97ed-42f9-8bb6-319f1698a233 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:38.851255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10003e98-ca68-48d3-857f-7e5bcc0ec097 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:38.851321 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddfdab-97ed-42f9-8bb6-319f1698a233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:38.851386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10003e98-ca68-48d3-857f-7e5bcc0ec097 None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:38.853974 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10003e98-ca68-48d3-857f-7e5bcc0ec097 None admin] RBAC: Authorizing `identity:get_project(project_id=8a3d0e211dd942aaa39eed1730d17e92)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:38.855497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10003e98-ca68-48d3-857f-7e5bcc0ec097 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:38.857349 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 97/186] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:38 2022] GET /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:38.858881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10003e98-ca68-48d3-857f-7e5bcc0ec097 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:38.867759 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:38.870895 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-f9ddfdab-97ed-42f9-8bb6-319f1698a233 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:38.872107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f9ddfdab-97ed-42f9-8bb6-319f1698a233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:38.953962 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 90/187] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:38.955386 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f1627-e721-4908-be1d-bf092adb6a83 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:38.955985 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f1627-e721-4908-be1d-bf092adb6a83 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:38.956358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f1627-e721-4908-be1d-bf092adb6a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:38.972368 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10003e98-ca68-48d3-857f-7e5bcc0ec097 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:38.972865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-572f1627-e721-4908-be1d-bf092adb6a83 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:38.974169 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-572f1627-e721-4908-be1d-bf092adb6a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:39.007575 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5fdfd0-f52b-42c1-a64f-55bf9e137313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:39.008244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5fdfd0-f52b-42c1-a64f-55bf9e137313 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:39.008533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5fdfd0-f52b-42c1-a64f-55bf9e137313 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:39.008816 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5fdfd0-f52b-42c1-a64f-55bf9e137313 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:39.009863 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5fdfd0-f52b-42c1-a64f-55bf9e137313 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:39.010176 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5fdfd0-f52b-42c1-a64f-55bf9e137313 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:39.011554 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5fdfd0-f52b-42c1-a64f-55bf9e137313 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:39.020301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 98/188] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:39.021097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5fdfd0-f52b-42c1-a64f-55bf9e137313 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:39.045704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 91/189] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:39.046546 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f1627-e721-4908-be1d-bf092adb6a83 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:39.083165 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:39.096781 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:39.132067 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5fdfd0-f52b-42c1-a64f-55bf9e137313 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:39.149528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f1627-e721-4908-be1d-bf092adb6a83 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:39.166732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b724626-f6e3-487a-961d-80e5c58c4d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:39.167389 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b724626-f6e3-487a-961d-80e5c58c4d33 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:39.167677 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b724626-f6e3-487a-961d-80e5c58c4d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:39.167959 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b724626-f6e3-487a-961d-80e5c58c4d33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:39.168802 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b724626-f6e3-487a-961d-80e5c58c4d33 None admin] RBAC: Adding query filter params (role.id=25a16d8ea4084e9eacdebb7a4dcd2337, scope.project.id=8a3d0e211dd942aaa39eed1730d17e92, user.id=eac0995e068547e3890aa6066bdecd84) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:39.169124 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b724626-f6e3-487a-961d-80e5c58c4d33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:39.170637 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b724626-f6e3-487a-961d-80e5c58c4d33 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:39.176643 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 99/190] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:20:39 2022] GET /identity/v3/role_assignments?user.id=eac0995e068547e3890aa6066bdecd84&scope.project.id=8a3d0e211dd942aaa39eed1730d17e92&role.id=25a16d8ea4084e9eacdebb7a4dcd2337 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:39.177488 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b724626-f6e3-487a-961d-80e5c58c4d33 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:39.197649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9de74-172b-463a-8c4c-d6537d8dd3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:39.198590 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9de74-172b-463a-8c4c-d6537d8dd3ed None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:39.198943 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9de74-172b-463a-8c4c-d6537d8dd3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:39.199296 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9de74-172b-463a-8c4c-d6537d8dd3ed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:39.212443 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9de74-172b-463a-8c4c-d6537d8dd3ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:39.220494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-dfe9de74-172b-463a-8c4c-d6537d8dd3ed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:20:39.220494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:39.220494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:39.220494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:39.220494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:39.220494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:39.220494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:39.220494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:39.220494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:39.220494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:39.220494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:39.220494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:39.220494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:39.220494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:39.220494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:39.220494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:39.222203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:39.222203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:39.222203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:39.222203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:39.222203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:39.222203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:39.222203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:39.222203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:39.222203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:39.222203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:39.222203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:39.222203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:39.222203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:39.222203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:39.222203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:39.222203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:39.223155 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:39.223155 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:39.223155 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:39.223155 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:39.223155 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:39.223155 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:39.223155 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:39.223155 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:39.223155 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:39.223155 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:39.223155 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:20:39.223155 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:20:39.223155 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 92/191] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:39.229534 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9de74-172b-463a-8c4c-d6537d8dd3ed None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:39.303581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b724626-f6e3-487a-961d-80e5c58c4d33 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:39.335931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:39.342020 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:39.343330 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f5c1d-f037-4d5c-b014-fef810334463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:39.344152 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f5c1d-f037-4d5c-b014-fef810334463 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:39.344432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f5c1d-f037-4d5c-b014-fef810334463 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:39.345130 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f5c1d-f037-4d5c-b014-fef810334463 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:39.353765 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9de74-172b-463a-8c4c-d6537d8dd3ed None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:39.360843 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f5c1d-f037-4d5c-b014-fef810334463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:39.369933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-371f5c1d-f037-4d5c-b014-fef810334463 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:20:39.369933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:39.369933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:39.369933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:39.369933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:39.369933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:39.369933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:39.369933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:39.369933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:39.369933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:39.369933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:39.369933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:39.369933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:39.369933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:39.369933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:39.369933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:39.370902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:39.370902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:39.370902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:39.370902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:39.370902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:39.370902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:39.370902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:39.370902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:39.370902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:39.370902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:39.370902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:39.370902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:39.370902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:39.370902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:39.370902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:39.370902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:39.371732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:39.371732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:39.371732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:39.371732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:39.371732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:39.371732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:39.371732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:39.371732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:39.371732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:39.371732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:39.371732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:20:39.371732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:20:39.371732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 100/192] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:20:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:39.373133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f5c1d-f037-4d5c-b014-fef810334463 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:39.389096 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de556692-706a-4004-8e52-00236f86577e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:39.390016 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de556692-706a-4004-8e52-00236f86577e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:39.390303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de556692-706a-4004-8e52-00236f86577e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:39.390581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de556692-706a-4004-8e52-00236f86577e None admin] PATH_INFO: `/v3/users/fe5a62f54a5d46bf911f78bddcffb61f` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:39.405051 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de556692-706a-4004-8e52-00236f86577e None admin] RBAC: Authorizing `identity:get_user(user_id=fe5a62f54a5d46bf911f78bddcffb61f)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:39.406583 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de556692-706a-4004-8e52-00236f86577e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:39.408383 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 93/193] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:39 2022] GET /identity/v3/users/fe5a62f54a5d46bf911f78bddcffb61f => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:39.409332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de556692-706a-4004-8e52-00236f86577e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:39.485360 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f5c1d-f037-4d5c-b014-fef810334463 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:39.508117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de556692-706a-4004-8e52-00236f86577e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:39.520002 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3571708-500f-4c38-9599-67e5f44bbead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:39.520776 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3571708-500f-4c38-9599-67e5f44bbead None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:39.521068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3571708-500f-4c38-9599-67e5f44bbead None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:39.521385 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3571708-500f-4c38-9599-67e5f44bbead None admin] PATH_INFO: `/v3/domains/default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:39.522569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3571708-500f-4c38-9599-67e5f44bbead None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:39.524046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3571708-500f-4c38-9599-67e5f44bbead None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:39.525709 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 101/194] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:20:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:39.526471 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3571708-500f-4c38-9599-67e5f44bbead None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:39.547027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1ec51-3cea-4e6a-bed8-ca25ba878629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:39.547710 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1ec51-3cea-4e6a-bed8-ca25ba878629 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:39.548003 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1ec51-3cea-4e6a-bed8-ca25ba878629 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:39.548287 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1ec51-3cea-4e6a-bed8-ca25ba878629 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:39.549168 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1ec51-3cea-4e6a-bed8-ca25ba878629 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:39.549581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1ec51-3cea-4e6a-bed8-ca25ba878629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:39.551356 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1ec51-3cea-4e6a-bed8-ca25ba878629 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:39.560868 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 94/195] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:20:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:39.561731 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1ec51-3cea-4e6a-bed8-ca25ba878629 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:39.583101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:39.596660 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:39.640601 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3571708-500f-4c38-9599-67e5f44bbead None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:39.668309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1ec51-3cea-4e6a-bed8-ca25ba878629 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:39.674747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbfadf-dee5-4641-b339-465c3db3794d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:39.675535 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbfadf-dee5-4641-b339-465c3db3794d None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:39.675715 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbfadf-dee5-4641-b339-465c3db3794d None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:39.675993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbfadf-dee5-4641-b339-465c3db3794d None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:39.676756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbfadf-dee5-4641-b339-465c3db3794d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:39.677065 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbfadf-dee5-4641-b339-465c3db3794d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:39.678519 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbfadf-dee5-4641-b339-465c3db3794d None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:39.686632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 102/196] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:20:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:39.687580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbfadf-dee5-4641-b339-465c3db3794d None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:39.713233 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65eee9-483a-4e0e-b791-eb3b8c8de858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:39.713903 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65eee9-483a-4e0e-b791-eb3b8c8de858 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:39.714196 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65eee9-483a-4e0e-b791-eb3b8c8de858 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:39.714481 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65eee9-483a-4e0e-b791-eb3b8c8de858 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:39.790467 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65eee9-483a-4e0e-b791-eb3b8c8de858 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:39.791956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65eee9-483a-4e0e-b791-eb3b8c8de858 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:39.796146 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbfadf-dee5-4641-b339-465c3db3794d None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:39.809882 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:39.819663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:39.836099 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08556785-7fdb-46d2-be7c-39c8fd3d8909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:39.836783 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08556785-7fdb-46d2-be7c-39c8fd3d8909 None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:39.837077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08556785-7fdb-46d2-be7c-39c8fd3d8909 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:39.837421 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08556785-7fdb-46d2-be7c-39c8fd3d8909 None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:39.838181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08556785-7fdb-46d2-be7c-39c8fd3d8909 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:39.839646 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08556785-7fdb-46d2-be7c-39c8fd3d8909 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:39.873182 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 103/197] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 17:20:39 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:20:39.873182 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08556785-7fdb-46d2-be7c-39c8fd3d8909 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:39.942738 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 95/198] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:20:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:20:39.944286 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65eee9-483a-4e0e-b791-eb3b8c8de858 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:40.020442 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08556785-7fdb-46d2-be7c-39c8fd3d8909 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:40.066761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00ca7f-7537-4680-b789-43bf41a81b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:40.067689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00ca7f-7537-4680-b789-43bf41a81b71 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:40.068057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00ca7f-7537-4680-b789-43bf41a81b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:40.069196 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00ca7f-7537-4680-b789-43bf41a81b71 None admin] PATH_INFO: `/v3/users/a22703a1b8784cb888c43670d59fb559` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:40.069782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc00ca7f-7537-4680-b789-43bf41a81b71 None admin] RBAC: Authorizing `identity:get_user(user_id=a22703a1b8784cb888c43670d59fb559)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:40.073868 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc00ca7f-7537-4680-b789-43bf41a81b71 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:40.074440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 104/199] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:39 2022] GET /identity/v3/users/a22703a1b8784cb888c43670d59fb559 => generated 289 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:40.075281 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00ca7f-7537-4680-b789-43bf41a81b71 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:40.087556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65eee9-483a-4e0e-b791-eb3b8c8de858 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:40.108004 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:40.134669 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a1484-0bbe-40d1-b3fe-7ba9c6b51b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:40.135757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a1484-0bbe-40d1-b3fe-7ba9c6b51b10 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:40.136143 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a1484-0bbe-40d1-b3fe-7ba9c6b51b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:40.136524 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a1484-0bbe-40d1-b3fe-7ba9c6b51b10 None admin] PATH_INFO: `/v3/users/fe5a62f54a5d46bf911f78bddcffb61f` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:40.137980 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a1484-0bbe-40d1-b3fe-7ba9c6b51b10 None admin] RBAC: Authorizing `identity:get_user(user_id=fe5a62f54a5d46bf911f78bddcffb61f)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:40.139660 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a1484-0bbe-40d1-b3fe-7ba9c6b51b10 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:40.143102 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 96/200] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:39 2022] GET /identity/v3/users/fe5a62f54a5d46bf911f78bddcffb61f => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:40.144137 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a1484-0bbe-40d1-b3fe-7ba9c6b51b10 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:40.154734 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:40.246904 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00ca7f-7537-4680-b789-43bf41a81b71 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:40.293870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7139e-e599-4810-90df-95f85a80f1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:40.295174 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7139e-e599-4810-90df-95f85a80f1a2 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:40.295471 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7139e-e599-4810-90df-95f85a80f1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:40.295752 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7139e-e599-4810-90df-95f85a80f1a2 None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:40.297248 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7139e-e599-4810-90df-95f85a80f1a2 None admin] RBAC: Authorizing `identity:get_project(project_id=d2415dcdc76d43c486190d67433cf01b)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:40.298494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a1484-0bbe-40d1-b3fe-7ba9c6b51b10 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:40.298719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7139e-e599-4810-90df-95f85a80f1a2 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:40.301101 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 105/201] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:40 2022] GET /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:40.302895 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7139e-e599-4810-90df-95f85a80f1a2 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:40.351042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3acdf-9392-4232-9c45-9adcaaaab30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:40.351653 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3acdf-9392-4232-9c45-9adcaaaab30c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:40.351940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3acdf-9392-4232-9c45-9adcaaaab30c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:40.352224 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3acdf-9392-4232-9c45-9adcaaaab30c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:40.409667 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:40.434126 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3acdf-9392-4232-9c45-9adcaaaab30c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:40.436850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3acdf-9392-4232-9c45-9adcaaaab30c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:40.438228 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7139e-e599-4810-90df-95f85a80f1a2 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:40.459414 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:40.490632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb507f6-a117-4ffb-84c5-f3d9daf69c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:40.491910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb507f6-a117-4ffb-84c5-f3d9daf69c39 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:40.492761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb507f6-a117-4ffb-84c5-f3d9daf69c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:40.493270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb507f6-a117-4ffb-84c5-f3d9daf69c39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:40.557041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 97/202] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:20:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:20:40.558210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3acdf-9392-4232-9c45-9adcaaaab30c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:40.583421 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb507f6-a117-4ffb-84c5-f3d9daf69c39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:40.585471 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb507f6-a117-4ffb-84c5-f3d9daf69c39 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:40.671282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:40.694856 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3acdf-9392-4232-9c45-9adcaaaab30c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:40.717804 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 106/203] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:20:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:20:40.719006 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb507f6-a117-4ffb-84c5-f3d9daf69c39 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:40.735133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:40.748871 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70b722-64b6-4c8f-ad6a-5f602822bb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:40.750123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70b722-64b6-4c8f-ad6a-5f602822bb13 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:40.750715 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70b722-64b6-4c8f-ad6a-5f602822bb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:40.751225 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70b722-64b6-4c8f-ad6a-5f602822bb13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:40.753097 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70b722-64b6-4c8f-ad6a-5f602822bb13 None admin] RBAC: Adding query filter params (role.id=25a16d8ea4084e9eacdebb7a4dcd2337, scope.project.id=d2415dcdc76d43c486190d67433cf01b, user.id=fe5a62f54a5d46bf911f78bddcffb61f) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:40.753684 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70b722-64b6-4c8f-ad6a-5f602822bb13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:40.755557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70b722-64b6-4c8f-ad6a-5f602822bb13 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:40.762642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 98/204] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:20:40 2022] GET /identity/v3/role_assignments?user.id=fe5a62f54a5d46bf911f78bddcffb61f&scope.project.id=d2415dcdc76d43c486190d67433cf01b&role.id=25a16d8ea4084e9eacdebb7a4dcd2337 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:40.763842 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70b722-64b6-4c8f-ad6a-5f602822bb13 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:40.830915 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb507f6-a117-4ffb-84c5-f3d9daf69c39 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:40.871648 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de18c3c-1562-4408-b973-e9a9ae41e3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:40.879050 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de18c3c-1562-4408-b973-e9a9ae41e3ec None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:40.879373 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de18c3c-1562-4408-b973-e9a9ae41e3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:40.879646 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de18c3c-1562-4408-b973-e9a9ae41e3ec None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:40.882052 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de18c3c-1562-4408-b973-e9a9ae41e3ec None admin] RBAC: Authorizing `identity:get_project(project_id=8a3d0e211dd942aaa39eed1730d17e92)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:40.886217 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de18c3c-1562-4408-b973-e9a9ae41e3ec None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:40.891175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 107/205] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:40 2022] GET /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:40.896983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de18c3c-1562-4408-b973-e9a9ae41e3ec None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:40.899840 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70b722-64b6-4c8f-ad6a-5f602822bb13 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:40.953048 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75054288-8534-4df7-ab85-036b7b85f3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:40.953931 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75054288-8534-4df7-ab85-036b7b85f3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:40.954225 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75054288-8534-4df7-ab85-036b7b85f3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:40.954509 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75054288-8534-4df7-ab85-036b7b85f3c0 None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:40.956373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75054288-8534-4df7-ab85-036b7b85f3c0 None admin] RBAC: Authorizing `identity:get_project(project_id=d2415dcdc76d43c486190d67433cf01b)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:40.958883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75054288-8534-4df7-ab85-036b7b85f3c0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:40.961009 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 99/206] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:40 2022] GET /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:40.968285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75054288-8534-4df7-ab85-036b7b85f3c0 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:41.012839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:41.048272 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:41.074901 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de18c3c-1562-4408-b973-e9a9ae41e3ec None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:41.146116 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75054288-8534-4df7-ab85-036b7b85f3c0 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:41.158081 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95eb6a-39d1-48af-a434-a0b96e312cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:41.164220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95eb6a-39d1-48af-a434-a0b96e312cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:41.164220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95eb6a-39d1-48af-a434-a0b96e312cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:41.164220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95eb6a-39d1-48af-a434-a0b96e312cf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:41.164220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95eb6a-39d1-48af-a434-a0b96e312cf4 None admin] RBAC: Adding query filter params (role.id=075fc2a9c4184b9db26b044f3d963233, scope.project.id=8a3d0e211dd942aaa39eed1730d17e92, user.id=a22703a1b8784cb888c43670d59fb559) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:41.164220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95eb6a-39d1-48af-a434-a0b96e312cf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:41.164220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95eb6a-39d1-48af-a434-a0b96e312cf4 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:41.169437 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 108/207] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:20:40 2022] GET /identity/v3/role_assignments?user.id=a22703a1b8784cb888c43670d59fb559&scope.project.id=8a3d0e211dd942aaa39eed1730d17e92&role.id=075fc2a9c4184b9db26b044f3d963233 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:41.224968 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca4273-ce5c-48a4-8078-99cd1bebd821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:41.225615 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca4273-ce5c-48a4-8078-99cd1bebd821 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:41.225906 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca4273-ce5c-48a4-8078-99cd1bebd821 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:41.226187 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca4273-ce5c-48a4-8078-99cd1bebd821 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:41.227059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca4273-ce5c-48a4-8078-99cd1bebd821 None admin] RBAC: Adding query filter params (role.id=1423a24997e840608b6c522958481fde, scope.project.id=d2415dcdc76d43c486190d67433cf01b, user.id=fe5a62f54a5d46bf911f78bddcffb61f) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:41.227383 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca4273-ce5c-48a4-8078-99cd1bebd821 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:41.228886 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca4273-ce5c-48a4-8078-99cd1bebd821 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:41.249199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 100/208] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:20:40 2022] GET /identity/v3/role_assignments?user.id=fe5a62f54a5d46bf911f78bddcffb61f&scope.project.id=d2415dcdc76d43c486190d67433cf01b&role.id=1423a24997e840608b6c522958481fde => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:42.349602 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d63b1-d047-4bc5-8822-c9f3a49f262a None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:42.350645 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d63b1-d047-4bc5-8822-c9f3a49f262a None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:42.351139 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d63b1-d047-4bc5-8822-c9f3a49f262a None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:42.352839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 109/209] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:42.358944 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd8c25-269b-41cb-a794-cd180b5df328 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:42.359939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd8c25-269b-41cb-a794-cd180b5df328 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:42.360418 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd8c25-269b-41cb-a794-cd180b5df328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:42.379461 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-56cd8c25-269b-41cb-a794-cd180b5df328 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:42.381042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-56cd8c25-269b-41cb-a794-cd180b5df328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:42.409723 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dba3c0-7d83-4dc6-b153-096a5113058a None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:42.410052 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dba3c0-7d83-4dc6-b153-096a5113058a None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:42.410310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dba3c0-7d83-4dc6-b153-096a5113058a None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:42.412108 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 110/210] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:20:42.416844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb06257-72d9-4140-8b14-c309b06d3275 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:42.417141 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb06257-72d9-4140-8b14-c309b06d3275 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:42.417459 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb06257-72d9-4140-8b14-c309b06d3275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:42.434357 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-ecb06257-72d9-4140-8b14-c309b06d3275 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:42.435561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-ecb06257-72d9-4140-8b14-c309b06d3275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:42.463484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 101/211] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:42.486869 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd8c25-269b-41cb-a794-cd180b5df328 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:42.509682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 111/212] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:20:42.521503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:42.534846 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb06257-72d9-4140-8b14-c309b06d3275 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:42.589505 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd8c25-269b-41cb-a794-cd180b5df328 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:42.616007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:42.625081 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6536c6a-cad7-43d1-8aa9-1610f7042b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:42.626229 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6536c6a-cad7-43d1-8aa9-1610f7042b22 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:42.626711 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6536c6a-cad7-43d1-8aa9-1610f7042b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:42.627771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6536c6a-cad7-43d1-8aa9-1610f7042b22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:42.638724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6536c6a-cad7-43d1-8aa9-1610f7042b22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:42.645007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb06257-72d9-4140-8b14-c309b06d3275 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:42.647124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-f6536c6a-cad7-43d1-8aa9-1610f7042b22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:42.647124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:42.647124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:42.647124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:42.647124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:42.647124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:42.647124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:42.647124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:42.647124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:42.647124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:42.647124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:42.647124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:42.647124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:42.647124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:42.647124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:42.647124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:42.647989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:42.647989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:42.647989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:42.647989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:42.647989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:42.647989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:42.647989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:42.647989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:42.647989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:42.647989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:42.647989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:42.647989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:42.647989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:42.647989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:42.647989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:42.647989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:42.648813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:42.648813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:42.648813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:42.648813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:42.648813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:42.648813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:42.648813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:42.648813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:42.648813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:42.648813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:42.648813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:42.648813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:20:42.657956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 102/213] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:20:42.659564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6536c6a-cad7-43d1-8aa9-1610f7042b22 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:42.681921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036cd0a3-104e-492c-85bb-3f65d443a994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:42.682414 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cd0a3-104e-492c-85bb-3f65d443a994 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:42.682707 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cd0a3-104e-492c-85bb-3f65d443a994 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:42.682994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cd0a3-104e-492c-85bb-3f65d443a994 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:42.684185 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036cd0a3-104e-492c-85bb-3f65d443a994 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:42.685695 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036cd0a3-104e-492c-85bb-3f65d443a994 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:42.692548 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 112/214] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:20:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:42.693770 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036cd0a3-104e-492c-85bb-3f65d443a994 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:42.757595 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:42.782842 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6536c6a-cad7-43d1-8aa9-1610f7042b22 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:42.808552 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036cd0a3-104e-492c-85bb-3f65d443a994 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:42.821589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9ef8a-4d8d-4ea1-81c9-b63a491bc643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:42.822427 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9ef8a-4d8d-4ea1-81c9-b63a491bc643 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:42.822815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9ef8a-4d8d-4ea1-81c9-b63a491bc643 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:42.823197 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9ef8a-4d8d-4ea1-81c9-b63a491bc643 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:42.824062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9ef8a-4d8d-4ea1-81c9-b63a491bc643 None admin] RBAC: Adding query filter params (name=member) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:42.824478 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9ef8a-4d8d-4ea1-81c9-b63a491bc643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:42.826009 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9ef8a-4d8d-4ea1-81c9-b63a491bc643 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:42.836101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 103/215] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:20:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:42.842307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9ef8a-4d8d-4ea1-81c9-b63a491bc643 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:42.852903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd088d7-e37f-4db2-8e29-4ce1ace5ec00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:42.853910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd088d7-e37f-4db2-8e29-4ce1ace5ec00 None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:42.854374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd088d7-e37f-4db2-8e29-4ce1ace5ec00 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:42.854821 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd088d7-e37f-4db2-8e29-4ce1ace5ec00 None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:42.855815 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd088d7-e37f-4db2-8e29-4ce1ace5ec00 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:42.857566 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd088d7-e37f-4db2-8e29-4ce1ace5ec00 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:42.879936 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:42.886877 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 113/216] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 17:20:42 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:20:42.969366 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9ef8a-4d8d-4ea1-81c9-b63a491bc643 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:43.004503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:43.011222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63c8c8-93c4-4847-a7d1-41caa7601803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:43.012399 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63c8c8-93c4-4847-a7d1-41caa7601803 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:43.013016 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63c8c8-93c4-4847-a7d1-41caa7601803 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:43.013484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63c8c8-93c4-4847-a7d1-41caa7601803 None admin] PATH_INFO: `/v3/users/b3b50699be764e628210e956b938cc0a` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:43.030450 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b63c8c8-93c4-4847-a7d1-41caa7601803 None admin] RBAC: Authorizing `identity:get_user(user_id=b3b50699be764e628210e956b938cc0a)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:43.032332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b63c8c8-93c4-4847-a7d1-41caa7601803 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:43.034814 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 104/217] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:42 2022] GET /identity/v3/users/b3b50699be764e628210e956b938cc0a => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:43.040715 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd088d7-e37f-4db2-8e29-4ce1ace5ec00 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:43.150467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd088d7-e37f-4db2-8e29-4ce1ace5ec00 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:43.186652 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfda5e2-b4e1-45a1-8343-c1487563bef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:43.187690 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfda5e2-b4e1-45a1-8343-c1487563bef2 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:43.188204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfda5e2-b4e1-45a1-8343-c1487563bef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:43.188688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfda5e2-b4e1-45a1-8343-c1487563bef2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:43.266411 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80f0ec-9b48-4ffd-afc3-ac3b0acafde3 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:43.267027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80f0ec-9b48-4ffd-afc3-ac3b0acafde3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:43.267505 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80f0ec-9b48-4ffd-afc3-ac3b0acafde3 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:43.268053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:43.269994 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 105/218] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:20:43.273605 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfda5e2-b4e1-45a1-8343-c1487563bef2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:43.273605 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfda5e2-b4e1-45a1-8343-c1487563bef2 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:43.275643 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cd702-5800-46e4-b59f-def1dfc8a040 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:43.276193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cd702-5800-46e4-b59f-def1dfc8a040 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:43.276696 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cd702-5800-46e4-b59f-def1dfc8a040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:43.301717 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d46cd702-5800-46e4-b59f-def1dfc8a040 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:43.303249 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d46cd702-5800-46e4-b59f-def1dfc8a040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:43.400168 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 106/219] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:43.401743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 114/220] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:20:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:20:43.408746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cd702-5800-46e4-b59f-def1dfc8a040 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:43.426849 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfda5e2-b4e1-45a1-8343-c1487563bef2 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:43.533010 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cd702-5800-46e4-b59f-def1dfc8a040 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:43.556471 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:43.565290 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfda5e2-b4e1-45a1-8343-c1487563bef2 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:43.572251 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96552ff8-30d7-465c-a7ee-3f21b50b5821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:43.573454 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96552ff8-30d7-465c-a7ee-3f21b50b5821 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:43.573937 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96552ff8-30d7-465c-a7ee-3f21b50b5821 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:43.574404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96552ff8-30d7-465c-a7ee-3f21b50b5821 None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:43.575802 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96552ff8-30d7-465c-a7ee-3f21b50b5821 None admin] RBAC: Authorizing `identity:get_project(project_id=d2415dcdc76d43c486190d67433cf01b)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:43.577578 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96552ff8-30d7-465c-a7ee-3f21b50b5821 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:43.579992 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 107/221] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:43 2022] GET /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:43.585508 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:43.587164 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e5148-e220-4547-a285-f1d5155447d3 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:43.587661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e5148-e220-4547-a285-f1d5155447d3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:43.588179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e5148-e220-4547-a285-f1d5155447d3 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:43.589798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 108/222] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:43.591101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e5148-e220-4547-a285-f1d5155447d3 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:43.608073 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c525aba-b1d4-4064-806f-0c7982015faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:43.608884 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c525aba-b1d4-4064-806f-0c7982015faf None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:43.609193 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c525aba-b1d4-4064-806f-0c7982015faf None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:43.609483 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c525aba-b1d4-4064-806f-0c7982015faf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:43.623821 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c525aba-b1d4-4064-806f-0c7982015faf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:43.635088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-2c525aba-b1d4-4064-806f-0c7982015faf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:20:43.635088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:43.635088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:43.635088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:43.635088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:43.635088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:43.635088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:43.635088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:43.635088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:43.635088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:43.635088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:43.635088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:43.635088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:43.635088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:43.635088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:43.635088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:43.636031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:43.636031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:43.636031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:43.636031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:43.636031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:43.636031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:43.636031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:43.636031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:43.636031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:43.636031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:43.636031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:43.636031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:43.636031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:43.636031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:43.636031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:43.636031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:43.636850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:43.636850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:43.636850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:43.636850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:43.636850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:43.636850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:43.636850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:43.636850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:43.636850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:43.636850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:43.636850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:20:43.636850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:20:43.636850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 115/223] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:20:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:43.641747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36506bb6-43c4-4712-b1d5-4921b4b91a05 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:43.641747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36506bb6-43c4-4712-b1d5-4921b4b91a05 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:43.641747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36506bb6-43c4-4712-b1d5-4921b4b91a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:43.658281 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-36506bb6-43c4-4712-b1d5-4921b4b91a05 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:43.659833 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-36506bb6-43c4-4712-b1d5-4921b4b91a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:43.717910 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e5148-e220-4547-a285-f1d5155447d3 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:43.746846 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 116/224] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:43.748753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36506bb6-43c4-4712-b1d5-4921b4b91a05 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:43.773179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b05709-a321-464f-bab7-0d0816f0c865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:43.773897 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b05709-a321-464f-bab7-0d0816f0c865 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:43.774181 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b05709-a321-464f-bab7-0d0816f0c865 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:43.774506 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b05709-a321-464f-bab7-0d0816f0c865 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:43.775347 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b05709-a321-464f-bab7-0d0816f0c865 None admin] RBAC: Adding query filter params (role.id=483146fdc4ac47cf97c04a9f66c091b4, scope.project.id=d2415dcdc76d43c486190d67433cf01b, user.id=b3b50699be764e628210e956b938cc0a) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:43.775664 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b05709-a321-464f-bab7-0d0816f0c865 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:43.778473 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b05709-a321-464f-bab7-0d0816f0c865 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:43.787379 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 109/225] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:20:43 2022] GET /identity/v3/role_assignments?user.id=b3b50699be764e628210e956b938cc0a&scope.project.id=d2415dcdc76d43c486190d67433cf01b&role.id=483146fdc4ac47cf97c04a9f66c091b4 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:43.788503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b05709-a321-464f-bab7-0d0816f0c865 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:43.895698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:43.928646 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:43.953738 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36506bb6-43c4-4712-b1d5-4921b4b91a05 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:43.995434 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b05709-a321-464f-bab7-0d0816f0c865 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:44.031396 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5b5e7-4d26-45eb-814d-edcc966bb4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:44.034894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5b5e7-4d26-45eb-814d-edcc966bb4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:44.035185 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5b5e7-4d26-45eb-814d-edcc966bb4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:44.037991 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5b5e7-4d26-45eb-814d-edcc966bb4e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:44.037991 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5b5e7-4d26-45eb-814d-edcc966bb4e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:44.037991 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5b5e7-4d26-45eb-814d-edcc966bb4e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:44.039784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5b5e7-4d26-45eb-814d-edcc966bb4e6 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:44.051057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 117/226] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:44.057396 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5b5e7-4d26-45eb-814d-edcc966bb4e6 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:44.060320 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf15a8-c08f-4653-a816-86f02988367f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:44.061423 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf15a8-c08f-4653-a816-86f02988367f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:44.061922 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf15a8-c08f-4653-a816-86f02988367f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:44.062392 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf15a8-c08f-4653-a816-86f02988367f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:44.074267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf15a8-c08f-4653-a816-86f02988367f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:44.083475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-70cf15a8-c08f-4653-a816-86f02988367f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:20:44.083475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:44.083475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:44.083475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:44.083475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:44.083475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:44.083475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:44.083475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:44.083475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:44.083475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:44.083475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:44.083475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:44.083475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:44.083475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:44.083475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:44.083475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:44.084412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:44.084412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:44.084412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:44.084412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:44.084412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:44.084412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:44.084412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:44.084412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:44.084412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:44.084412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:44.084412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:44.084412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:44.084412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:44.084412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:44.084412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:44.084412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:44.085281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:44.085281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:44.085281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:44.085281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:44.085281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:44.085281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:44.085281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:44.085281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:44.085281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:44.085281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:44.085281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:20:44.085281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:20:44.088015 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 110/227] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:20:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:44.093972 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf15a8-c08f-4653-a816-86f02988367f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:44.224394 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5b5e7-4d26-45eb-814d-edcc966bb4e6 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:44.274408 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf15a8-c08f-4653-a816-86f02988367f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:44.290915 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0254c5fa-b329-47d4-b2f2-d12cf322e03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:44.292108 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254c5fa-b329-47d4-b2f2-d12cf322e03c None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:44.294452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254c5fa-b329-47d4-b2f2-d12cf322e03c None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:44.294962 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254c5fa-b329-47d4-b2f2-d12cf322e03c None admin] PATH_INFO: `/v3/users/fe5a62f54a5d46bf911f78bddcffb61f` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:44.298608 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0254c5fa-b329-47d4-b2f2-d12cf322e03c None admin] RBAC: Authorizing `identity:get_user(user_id=fe5a62f54a5d46bf911f78bddcffb61f)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:44.300625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0254c5fa-b329-47d4-b2f2-d12cf322e03c None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:44.305541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:44.307894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 118/228] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:44 2022] GET /identity/v3/users/fe5a62f54a5d46bf911f78bddcffb61f => generated 283 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:44.313263 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0254c5fa-b329-47d4-b2f2-d12cf322e03c None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:44.348036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fbc98-e7a7-41d3-805a-31c19e73d290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:44.349071 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fbc98-e7a7-41d3-805a-31c19e73d290 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:44.350351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fbc98-e7a7-41d3-805a-31c19e73d290 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:44.353489 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fbc98-e7a7-41d3-805a-31c19e73d290 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:44.354580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0fbc98-e7a7-41d3-805a-31c19e73d290 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:44.355063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:44.355345 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0fbc98-e7a7-41d3-805a-31c19e73d290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:44.356932 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0fbc98-e7a7-41d3-805a-31c19e73d290 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:44.369080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 111/229] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:20:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:44.378748 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fbc98-e7a7-41d3-805a-31c19e73d290 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:44.458986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0254c5fa-b329-47d4-b2f2-d12cf322e03c None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:44.496823 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38873d05-8402-4b8f-9f37-503f094ff004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:44.498067 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38873d05-8402-4b8f-9f37-503f094ff004 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:44.498553 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38873d05-8402-4b8f-9f37-503f094ff004 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:44.499439 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38873d05-8402-4b8f-9f37-503f094ff004 None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:44.500842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38873d05-8402-4b8f-9f37-503f094ff004 None admin] RBAC: Authorizing `identity:get_project(project_id=d2415dcdc76d43c486190d67433cf01b)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:44.503278 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38873d05-8402-4b8f-9f37-503f094ff004 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:44.505906 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 119/230] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:44 2022] GET /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:44.511272 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38873d05-8402-4b8f-9f37-503f094ff004 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:44.632053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fbc98-e7a7-41d3-805a-31c19e73d290 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:44.661527 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:44.684979 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38873d05-8402-4b8f-9f37-503f094ff004 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:44.707505 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3522f3-e413-498f-a366-7ad8bd32a50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:44.709666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3522f3-e413-498f-a366-7ad8bd32a50f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:44.710738 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3522f3-e413-498f-a366-7ad8bd32a50f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:44.712267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3522f3-e413-498f-a366-7ad8bd32a50f None admin] PATH_INFO: `/v3/users/fe5a62f54a5d46bf911f78bddcffb61f` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:44.716941 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3522f3-e413-498f-a366-7ad8bd32a50f None admin] RBAC: Authorizing `identity:get_user(user_id=fe5a62f54a5d46bf911f78bddcffb61f)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:44.718656 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3522f3-e413-498f-a366-7ad8bd32a50f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:44.726445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 112/231] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:44 2022] GET /identity/v3/users/fe5a62f54a5d46bf911f78bddcffb61f => generated 283 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:44.730773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3522f3-e413-498f-a366-7ad8bd32a50f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:44.758577 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0d2a1-ac87-4a0c-a111-ccea8db46f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:44.760113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0d2a1-ac87-4a0c-a111-ccea8db46f8a None admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:44.760617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0d2a1-ac87-4a0c-a111-ccea8db46f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:44.761101 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0d2a1-ac87-4a0c-a111-ccea8db46f8a None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b/users/fe5a62f54a5d46bf911f78bddcffb61f/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:44.763832 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0d2a1-ac87-4a0c-a111-ccea8db46f8a None admin] RBAC: Authorizing `identity:create_grant(project_id=d2415dcdc76d43c486190d67433cf01b, user_id=fe5a62f54a5d46bf911f78bddcffb61f, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:44.765617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0d2a1-ac87-4a0c-a111-ccea8db46f8a None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:44.784375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 120/232] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:20:44 2022] PUT /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b/users/fe5a62f54a5d46bf911f78bddcffb61f/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:20:44.822457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:44.896619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3522f3-e413-498f-a366-7ad8bd32a50f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:44.943355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb59aba-060b-490c-b080-12d4079cb022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:44.944623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb59aba-060b-490c-b080-12d4079cb022 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:44.945126 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb59aba-060b-490c-b080-12d4079cb022 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:44.945619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb59aba-060b-490c-b080-12d4079cb022 None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:44.947003 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb59aba-060b-490c-b080-12d4079cb022 None admin] RBAC: Authorizing `identity:get_project(project_id=d2415dcdc76d43c486190d67433cf01b)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:44.948648 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb59aba-060b-490c-b080-12d4079cb022 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:44.950599 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 113/233] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:44 2022] GET /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:44.957568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0d2a1-ac87-4a0c-a111-ccea8db46f8a None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:45.077319 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0d2a1-ac87-4a0c-a111-ccea8db46f8a None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:45.113999 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e2f071-2cb8-40e8-b16f-c90e4ed803f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:45.115265 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2f071-2cb8-40e8-b16f-c90e4ed803f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:45.115769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2f071-2cb8-40e8-b16f-c90e4ed803f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:45.116244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2f071-2cb8-40e8-b16f-c90e4ed803f1 None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b/users/fe5a62f54a5d46bf911f78bddcffb61f/roles/1423a24997e840608b6c522958481fde` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:45.118703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2f071-2cb8-40e8-b16f-c90e4ed803f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=d2415dcdc76d43c486190d67433cf01b, user_id=fe5a62f54a5d46bf911f78bddcffb61f, role_id=1423a24997e840608b6c522958481fde)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:45.120359 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2f071-2cb8-40e8-b16f-c90e4ed803f1 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:45.149466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 121/234] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:20:44 2022] PUT /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b/users/fe5a62f54a5d46bf911f78bddcffb61f/roles/1423a24997e840608b6c522958481fde => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:20:45.508095 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e99e4f-a8c1-4521-81bf-6d701683c7cc None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:45.508743 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e99e4f-a8c1-4521-81bf-6d701683c7cc None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:45.509295 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e99e4f-a8c1-4521-81bf-6d701683c7cc None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:45.511388 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 114/235] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:45.526657 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5704434-4350-4b70-975e-c63670d31768 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:45.527598 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5704434-4350-4b70-975e-c63670d31768 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:45.528084 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5704434-4350-4b70-975e-c63670d31768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:45.540317 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:45.551036 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-c5704434-4350-4b70-975e-c63670d31768 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:45.553181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-c5704434-4350-4b70-975e-c63670d31768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:45.648220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 122/236] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:45.688977 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b4bca-af36-4ada-aa5f-fef485df80e5 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:45.689850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b4bca-af36-4ada-aa5f-fef485df80e5 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:45.689850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b4bca-af36-4ada-aa5f-fef485df80e5 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:45.691348 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 115/237] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:45.694698 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5704434-4350-4b70-975e-c63670d31768 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:45.712718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ce997-42fb-483e-b837-efeb68248e60 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:45.713032 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ce997-42fb-483e-b837-efeb68248e60 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:45.713310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ce997-42fb-483e-b837-efeb68248e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:45.740220 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-af7ce997-42fb-483e-b837-efeb68248e60 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:45.741450 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-af7ce997-42fb-483e-b837-efeb68248e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:45.839095 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5704434-4350-4b70-975e-c63670d31768 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:45.873717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df4649-89c9-4ace-800e-78a2bb4be75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:45.874583 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df4649-89c9-4ace-800e-78a2bb4be75d None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:45.874877 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df4649-89c9-4ace-800e-78a2bb4be75d None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:45.875206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df4649-89c9-4ace-800e-78a2bb4be75d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:45.886445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df4649-89c9-4ace-800e-78a2bb4be75d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:45.893590 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:45.898894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-98df4649-89c9-4ace-800e-78a2bb4be75d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:20:45.898894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:45.898894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:45.898894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:45.898894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:45.898894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:45.898894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:45.898894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:45.898894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:45.898894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:45.898894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:45.898894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:45.898894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:45.898894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:45.898894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:45.898894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:45.899813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:45.899813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:45.899813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:45.899813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:45.899813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:45.899813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:45.899813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:45.899813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:45.899813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:45.899813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:45.899813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:45.899813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:45.899813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:45.899813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:45.899813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:45.899813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:45.901051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:45.901051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:45.901051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:45.901051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:45.901051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:45.901051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:45.901051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:45.901051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:45.901051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:45.901051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:45.901051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:20:45.901051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:20:45.901051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 123/238] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:45.905988 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df4649-89c9-4ace-800e-78a2bb4be75d None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:45.915495 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 116/239] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:20:45.979352 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ce997-42fb-483e-b837-efeb68248e60 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:46.069553 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:46.129098 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ce997-42fb-483e-b837-efeb68248e60 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:46.155155 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df4649-89c9-4ace-800e-78a2bb4be75d None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:46.166032 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63850571-1431-4408-ba03-4a35eede9a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:46.166842 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63850571-1431-4408-ba03-4a35eede9a5f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:46.167127 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63850571-1431-4408-ba03-4a35eede9a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:46.167400 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63850571-1431-4408-ba03-4a35eede9a5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:46.168575 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63850571-1431-4408-ba03-4a35eede9a5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:46.170468 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63850571-1431-4408-ba03-4a35eede9a5f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:46.172516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 117/240] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:20:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:46.178032 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63850571-1431-4408-ba03-4a35eede9a5f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:46.265900 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d8e79-8e33-448a-93ec-7b6050c9d412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:46.266608 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d8e79-8e33-448a-93ec-7b6050c9d412 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:46.266901 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d8e79-8e33-448a-93ec-7b6050c9d412 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:46.267185 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d8e79-8e33-448a-93ec-7b6050c9d412 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:46.269886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d8e79-8e33-448a-93ec-7b6050c9d412 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:46.270214 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d8e79-8e33-448a-93ec-7b6050c9d412 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:46.271645 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d8e79-8e33-448a-93ec-7b6050c9d412 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:46.288483 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63850571-1431-4408-ba03-4a35eede9a5f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:46.298339 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 124/241] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:20:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:46.304613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d8e79-8e33-448a-93ec-7b6050c9d412 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:46.311342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:46.360088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d89ac-971d-4319-887e-4e2dd9e8ec99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:46.365774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d89ac-971d-4319-887e-4e2dd9e8ec99 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:46.366092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d89ac-971d-4319-887e-4e2dd9e8ec99 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:46.366362 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d89ac-971d-4319-887e-4e2dd9e8ec99 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:46.367139 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d89ac-971d-4319-887e-4e2dd9e8ec99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:46.368650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d89ac-971d-4319-887e-4e2dd9e8ec99 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:46.405377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 118/242] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 17:20:46 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:20:46.405721 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:46.437839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f8f7c-3b57-4d0d-95c6-bc107e8eb28d None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:46.438177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f8f7c-3b57-4d0d-95c6-bc107e8eb28d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:46.438439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f8f7c-3b57-4d0d-95c6-bc107e8eb28d None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:46.440257 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 119/243] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:20:46.447331 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5451911-9553-492b-a77b-99e06b878698 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:46.447637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5451911-9553-492b-a77b-99e06b878698 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:46.447902 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5451911-9553-492b-a77b-99e06b878698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:46.468831 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-f5451911-9553-492b-a77b-99e06b878698 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:46.470140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f5451911-9553-492b-a77b-99e06b878698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:46.492012 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d8e79-8e33-448a-93ec-7b6050c9d412 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:46.565115 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 120/244] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:20:46.579046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c3c04-6c11-4927-90aa-5fffafd43413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:46.579955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c3c04-6c11-4927-90aa-5fffafd43413 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:46.580467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c3c04-6c11-4927-90aa-5fffafd43413 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:46.581125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c3c04-6c11-4927-90aa-5fffafd43413 None admin] PATH_INFO: `/v3/users/5ad4c71102db46dfa8af097faf29ecf2` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:46.583725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5451911-9553-492b-a77b-99e06b878698 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:46.616378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c3c04-6c11-4927-90aa-5fffafd43413 None admin] RBAC: Authorizing `identity:get_user(user_id=5ad4c71102db46dfa8af097faf29ecf2)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:46.617937 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c3c04-6c11-4927-90aa-5fffafd43413 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:46.625572 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 125/245] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:46 2022] GET /identity/v3/users/5ad4c71102db46dfa8af097faf29ecf2 => generated 297 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:46.627702 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c3c04-6c11-4927-90aa-5fffafd43413 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:46.731619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:46.742797 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5451911-9553-492b-a77b-99e06b878698 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:46.758210 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c3c04-6c11-4927-90aa-5fffafd43413 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:46.777401 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:46.784824 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e926ea8-e331-4d0f-8022-d07d44c3f5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:46.785789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e926ea8-e331-4d0f-8022-d07d44c3f5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:46.786082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e926ea8-e331-4d0f-8022-d07d44c3f5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:46.786474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e926ea8-e331-4d0f-8022-d07d44c3f5e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:46.798103 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e926ea8-e331-4d0f-8022-d07d44c3f5e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:46.801327 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a6c46-45a2-4634-885a-c3245cdd385e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:46.801995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a6c46-45a2-4634-885a-c3245cdd385e None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:46.802290 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a6c46-45a2-4634-885a-c3245cdd385e None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:46.802580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a6c46-45a2-4634-885a-c3245cdd385e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:46.806393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-5e926ea8-e331-4d0f-8022-d07d44c3f5e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:46.806393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:46.806393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:46.806393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:46.806393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:46.806393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:46.806393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:46.806393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:46.806393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:46.806393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:46.806393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:46.806393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:46.806393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:46.806393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:46.806393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:46.806393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:46.807526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:46.807526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:46.807526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:46.807526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:46.807526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:46.807526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:46.807526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:46.807526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:46.807526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:46.807526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:46.807526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:46.807526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:46.807526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:46.807526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:46.807526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:46.807526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:46.808634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:46.808634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:46.808634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:46.808634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:46.808634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:46.808634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:46.808634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:46.808634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:46.808634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:46.808634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:46.808634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:46.808634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:20:46.812743 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 121/246] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:20:46.814016 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e926ea8-e331-4d0f-8022-d07d44c3f5e2 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:46.884913 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a6c46-45a2-4634-885a-c3245cdd385e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:46.886869 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a6c46-45a2-4634-885a-c3245cdd385e None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:46.953944 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e926ea8-e331-4d0f-8022-d07d44c3f5e2 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:46.997271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d95f88-98ef-43af-b0f1-3713419a81b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:46.997271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d95f88-98ef-43af-b0f1-3713419a81b7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:46.997271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d95f88-98ef-43af-b0f1-3713419a81b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:46.997271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d95f88-98ef-43af-b0f1-3713419a81b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:46.998393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d95f88-98ef-43af-b0f1-3713419a81b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:46.998706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d95f88-98ef-43af-b0f1-3713419a81b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:47.000094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d95f88-98ef-43af-b0f1-3713419a81b7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:47.012072 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 122/247] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:20:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:47.016472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d95f88-98ef-43af-b0f1-3713419a81b7 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:47.041200 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 126/248] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:20:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:20:47.046719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a6c46-45a2-4634-885a-c3245cdd385e None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:47.047405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:47.073665 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:47.151976 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d95f88-98ef-43af-b0f1-3713419a81b7 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:47.188626 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b75eb-ba5d-4353-953d-027b70274059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:47.189748 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b75eb-ba5d-4353-953d-027b70274059 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:47.193185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b75eb-ba5d-4353-953d-027b70274059 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:47.193185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b75eb-ba5d-4353-953d-027b70274059 None admin] PATH_INFO: `/v3/users/b3b50699be764e628210e956b938cc0a` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:47.193185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b75eb-ba5d-4353-953d-027b70274059 None admin] RBAC: Authorizing `identity:get_user(user_id=b3b50699be764e628210e956b938cc0a)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:47.193185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b75eb-ba5d-4353-953d-027b70274059 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:47.195125 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 123/249] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:47 2022] GET /identity/v3/users/b3b50699be764e628210e956b938cc0a => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:47.196786 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a6c46-45a2-4634-885a-c3245cdd385e None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:47.204153 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b75eb-ba5d-4353-953d-027b70274059 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:47.235393 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0175a-6c05-4a40-aff0-4a3faddf6ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:47.237305 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0175a-6c05-4a40-aff0-4a3faddf6ffe None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:47.237799 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0175a-6c05-4a40-aff0-4a3faddf6ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:47.238279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0175a-6c05-4a40-aff0-4a3faddf6ffe None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:47.239726 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac0175a-6c05-4a40-aff0-4a3faddf6ffe None admin] RBAC: Authorizing `identity:get_project(project_id=d2415dcdc76d43c486190d67433cf01b)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:47.242090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac0175a-6c05-4a40-aff0-4a3faddf6ffe None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:47.244606 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 127/250] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:47 2022] GET /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:47.264979 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0175a-6c05-4a40-aff0-4a3faddf6ffe None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:47.323764 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:47.349580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b75eb-ba5d-4353-953d-027b70274059 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:47.442630 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de066a-6250-4767-8ce3-0d10c084213d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:47.443527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de066a-6250-4767-8ce3-0d10c084213d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:47.443818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de066a-6250-4767-8ce3-0d10c084213d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:47.444100 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de066a-6250-4767-8ce3-0d10c084213d None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:47.446536 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de066a-6250-4767-8ce3-0d10c084213d None admin] RBAC: Authorizing `identity:get_project(project_id=d2415dcdc76d43c486190d67433cf01b)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:47.448014 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de066a-6250-4767-8ce3-0d10c084213d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:47.450976 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 124/251] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:47 2022] GET /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:47.459113 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de066a-6250-4767-8ce3-0d10c084213d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:47.466917 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:47.500617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0175a-6c05-4a40-aff0-4a3faddf6ffe None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:47.579162 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954a1ff-48f4-46a7-bdf3-c4a9528d5024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:47.579831 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954a1ff-48f4-46a7-bdf3-c4a9528d5024 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:47.580216 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954a1ff-48f4-46a7-bdf3-c4a9528d5024 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:47.580887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954a1ff-48f4-46a7-bdf3-c4a9528d5024 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:47.582099 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954a1ff-48f4-46a7-bdf3-c4a9528d5024 None admin] RBAC: Adding query filter params (role.id=075fc2a9c4184b9db26b044f3d963233, scope.project.id=d2415dcdc76d43c486190d67433cf01b, user.id=5ad4c71102db46dfa8af097faf29ecf2) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:47.582431 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954a1ff-48f4-46a7-bdf3-c4a9528d5024 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:47.583919 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954a1ff-48f4-46a7-bdf3-c4a9528d5024 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:47.593487 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 128/252] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:20:47 2022] GET /identity/v3/role_assignments?user.id=5ad4c71102db46dfa8af097faf29ecf2&scope.project.id=d2415dcdc76d43c486190d67433cf01b&role.id=075fc2a9c4184b9db26b044f3d963233 => generated 259 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:47.672174 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de066a-6250-4767-8ce3-0d10c084213d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:47.723516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112c2bd-bf2b-4684-a458-b8b4a22803be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:47.724452 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112c2bd-bf2b-4684-a458-b8b4a22803be None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:47.724740 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112c2bd-bf2b-4684-a458-b8b4a22803be None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:47.725020 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112c2bd-bf2b-4684-a458-b8b4a22803be None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b/users/b3b50699be764e628210e956b938cc0a/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:47.727225 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3112c2bd-bf2b-4684-a458-b8b4a22803be None admin] RBAC: Authorizing `identity:create_grant(project_id=d2415dcdc76d43c486190d67433cf01b, user_id=b3b50699be764e628210e956b938cc0a, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:47.728734 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3112c2bd-bf2b-4684-a458-b8b4a22803be None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:47.743677 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 125/253] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:20:47 2022] PUT /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b/users/b3b50699be764e628210e956b938cc0a/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:20:47.851824 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a27b4-cd07-4aba-8b53-7ea39be878c1 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:47.852438 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a27b4-cd07-4aba-8b53-7ea39be878c1 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:47.852922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a27b4-cd07-4aba-8b53-7ea39be878c1 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:47.854612 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 129/254] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:47.860482 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf9abb-c1b7-4743-bf70-9cf4894fc638 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:47.860800 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf9abb-c1b7-4743-bf70-9cf4894fc638 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:47.861056 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf9abb-c1b7-4743-bf70-9cf4894fc638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:47.889557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-6dcf9abb-c1b7-4743-bf70-9cf4894fc638 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:47.890803 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6dcf9abb-c1b7-4743-bf70-9cf4894fc638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:47.904206 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:47.982050 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 126/255] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:48.020873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a27b4-cd07-4aba-8b53-7ea39be878c1 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:48.047808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e339bbe-ff49-43e6-b2ab-6821dea37e4f None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:48.048135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e339bbe-ff49-43e6-b2ab-6821dea37e4f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:48.048384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e339bbe-ff49-43e6-b2ab-6821dea37e4f None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:48.050824 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 127/256] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:48.058413 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce3f07-6fcd-4505-8b57-dec8a6ba8507 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:48.058701 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce3f07-6fcd-4505-8b57-dec8a6ba8507 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:48.058995 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce3f07-6fcd-4505-8b57-dec8a6ba8507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:48.089853 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e2ce3f07-6fcd-4505-8b57-dec8a6ba8507 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:48.091078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e2ce3f07-6fcd-4505-8b57-dec8a6ba8507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:48.187123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 128/257] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:20:48.197276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a27b4-cd07-4aba-8b53-7ea39be878c1 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:48.227760 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce3f07-6fcd-4505-8b57-dec8a6ba8507 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:48.263148 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d4bb8-1f5e-4b83-b3af-41fccb62a185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:48.263961 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d4bb8-1f5e-4b83-b3af-41fccb62a185 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:48.264255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d4bb8-1f5e-4b83-b3af-41fccb62a185 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:48.264546 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d4bb8-1f5e-4b83-b3af-41fccb62a185 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:48.292830 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d4bb8-1f5e-4b83-b3af-41fccb62a185 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:48.304949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-9c4d4bb8-1f5e-4b83-b3af-41fccb62a185 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:20:48.304949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:48.304949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:48.304949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:48.304949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:48.304949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:48.304949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:48.304949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:48.304949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:48.304949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:48.304949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:48.304949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:48.304949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:48.304949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:48.304949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:48.304949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:48.305889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:48.305889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:48.305889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:48.305889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:48.305889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:48.305889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:48.305889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:48.305889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:48.305889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:48.305889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:48.305889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:48.305889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:48.305889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:48.305889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:48.305889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:48.305889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:48.306719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:48.306719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:48.306719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:48.306719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:48.306719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:48.306719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:48.306719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:48.306719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:48.306719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:48.306719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:48.306719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:20:48.306719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:20:48.306719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 130/258] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:20:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:48.315942 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d4bb8-1f5e-4b83-b3af-41fccb62a185 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:48.338594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:48.374653 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:48.431064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce3f07-6fcd-4505-8b57-dec8a6ba8507 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:48.494551 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d4bb8-1f5e-4b83-b3af-41fccb62a185 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:48.520782 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c3907-2cd9-48b9-94d1-f2e233a854a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:48.521641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c3907-2cd9-48b9-94d1-f2e233a854a7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:48.521987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c3907-2cd9-48b9-94d1-f2e233a854a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:48.522272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c3907-2cd9-48b9-94d1-f2e233a854a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:48.542018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ad1b4-31bc-496d-aae6-900df25a42a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:48.543056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ad1b4-31bc-496d-aae6-900df25a42a3 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:48.543501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ad1b4-31bc-496d-aae6-900df25a42a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:48.543920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ad1b4-31bc-496d-aae6-900df25a42a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:48.544981 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c3907-2cd9-48b9-94d1-f2e233a854a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:48.545121 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ad1b4-31bc-496d-aae6-900df25a42a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:48.545916 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ad1b4-31bc-496d-aae6-900df25a42a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:48.548292 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ad1b4-31bc-496d-aae6-900df25a42a3 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:48.554520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-d49c3907-2cd9-48b9-94d1-f2e233a854a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:20:48.554520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:48.554520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:48.554520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:48.554520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:48.554520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:48.554520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:48.554520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:48.554520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:48.554520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:48.554520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:48.554520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:48.554520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:48.554520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:48.554520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:48.554520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:48.555417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:48.555417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:48.555417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:48.555417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:48.555417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:48.555417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:48.555417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:48.555417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:48.555417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:48.555417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:48.555417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:48.555417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:48.555417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:48.555417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:48.555417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:48.555417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:48.556307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:48.556307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:48.556307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:48.556307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:48.556307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:48.556307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:48.556307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:48.556307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:48.556307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:48.556307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:48.556307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:20:48.556307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:20:48.559319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 129/259] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:20:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:20:48.566294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 131/260] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:48.571509 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c3907-2cd9-48b9-94d1-f2e233a854a7 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:48.571942 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ad1b4-31bc-496d-aae6-900df25a42a3 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:48.651538 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:48.678033 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ad1b4-31bc-496d-aae6-900df25a42a3 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:48.716185 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f90cd-1725-4d74-84b4-60c004aa178b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:48.716897 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f90cd-1725-4d74-84b4-60c004aa178b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:48.717220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f90cd-1725-4d74-84b4-60c004aa178b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:48.717511 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f90cd-1725-4d74-84b4-60c004aa178b None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:48.718305 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f90cd-1725-4d74-84b4-60c004aa178b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:48.718623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f90cd-1725-4d74-84b4-60c004aa178b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:48.720196 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f90cd-1725-4d74-84b4-60c004aa178b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:48.729877 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 132/261] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:20:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:48.735631 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f90cd-1725-4d74-84b4-60c004aa178b None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:48.740008 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c3907-2cd9-48b9-94d1-f2e233a854a7 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:48.810496 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:48.825258 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca11a0-f2ab-4293-b0bd-51bfd3eed061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:48.826518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca11a0-f2ab-4293-b0bd-51bfd3eed061 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:48.827000 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca11a0-f2ab-4293-b0bd-51bfd3eed061 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:48.827506 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca11a0-f2ab-4293-b0bd-51bfd3eed061 None admin] PATH_INFO: `/v3/users/fe5a62f54a5d46bf911f78bddcffb61f` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:48.828996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca11a0-f2ab-4293-b0bd-51bfd3eed061 None admin] RBAC: Authorizing `identity:get_user(user_id=fe5a62f54a5d46bf911f78bddcffb61f)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:48.830853 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca11a0-f2ab-4293-b0bd-51bfd3eed061 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:48.833139 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 130/262] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:48 2022] GET /identity/v3/users/fe5a62f54a5d46bf911f78bddcffb61f => generated 283 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:48.838553 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca11a0-f2ab-4293-b0bd-51bfd3eed061 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:48.915884 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f90cd-1725-4d74-84b4-60c004aa178b None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:48.961134 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:48.968846 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb0309-77f5-4ed8-8de8-c5e50e96ce1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:48.970083 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb0309-77f5-4ed8-8de8-c5e50e96ce1b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:48.971109 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb0309-77f5-4ed8-8de8-c5e50e96ce1b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:48.971751 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb0309-77f5-4ed8-8de8-c5e50e96ce1b None admin] PATH_INFO: `/v3/users/fe5a62f54a5d46bf911f78bddcffb61f` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:48.973447 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb0309-77f5-4ed8-8de8-c5e50e96ce1b None admin] RBAC: Authorizing `identity:get_user(user_id=fe5a62f54a5d46bf911f78bddcffb61f)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:48.975379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb0309-77f5-4ed8-8de8-c5e50e96ce1b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:48.977345 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 133/263] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:48 2022] GET /identity/v3/users/fe5a62f54a5d46bf911f78bddcffb61f => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:48.982951 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb0309-77f5-4ed8-8de8-c5e50e96ce1b None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:49.097454 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb0309-77f5-4ed8-8de8-c5e50e96ce1b None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:49.103080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca11a0-f2ab-4293-b0bd-51bfd3eed061 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:49.136171 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63518d32-f5c4-4243-9abf-1d4800e892cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:49.137625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63518d32-f5c4-4243-9abf-1d4800e892cf None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:49.138013 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63518d32-f5c4-4243-9abf-1d4800e892cf None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:49.138380 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63518d32-f5c4-4243-9abf-1d4800e892cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:49.213563 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:49.217831 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63518d32-f5c4-4243-9abf-1d4800e892cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:49.219402 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63518d32-f5c4-4243-9abf-1d4800e892cf None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:49.230492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9ba74-5a98-4a2a-b196-76efe2e80218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:49.231178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9ba74-5a98-4a2a-b196-76efe2e80218 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:49.231469 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9ba74-5a98-4a2a-b196-76efe2e80218 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:49.231746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9ba74-5a98-4a2a-b196-76efe2e80218 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:49.444509 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 134/264] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:20:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:20:49.450274 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63518d32-f5c4-4243-9abf-1d4800e892cf None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:49.502980 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:49.506609 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c9ba74-5a98-4a2a-b196-76efe2e80218 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:49.508127 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c9ba74-5a98-4a2a-b196-76efe2e80218 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:49.644855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 131/265] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:20:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 806 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:20:49.654261 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9ba74-5a98-4a2a-b196-76efe2e80218 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:49.835610 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63518d32-f5c4-4243-9abf-1d4800e892cf None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:49.837688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:49.841426 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9ba74-5a98-4a2a-b196-76efe2e80218 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:49.843432 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:49.885613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9621d-21d1-4682-ae12-57714a4fcef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:49.886482 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9621d-21d1-4682-ae12-57714a4fcef1 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:49.886772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9621d-21d1-4682-ae12-57714a4fcef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:49.887055 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9621d-21d1-4682-ae12-57714a4fcef1 None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:49.888257 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9621d-21d1-4682-ae12-57714a4fcef1 None admin] RBAC: Authorizing `identity:get_project(project_id=d2415dcdc76d43c486190d67433cf01b)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:49.889748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9621d-21d1-4682-ae12-57714a4fcef1 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:49.891488 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 135/266] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:49 2022] GET /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b => generated 302 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:49.892745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5df26b-2185-47e6-b553-e3a293859683 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:49.893038 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5df26b-2185-47e6-b553-e3a293859683 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:49.893385 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5df26b-2185-47e6-b553-e3a293859683 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:49.894628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 136/267] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:49.899854 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a48038-4244-4958-a38e-05d41c66e46f None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:49.900152 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a48038-4244-4958-a38e-05d41c66e46f None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:49.900442 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a48038-4244-4958-a38e-05d41c66e46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:49.913936 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469206d-4a38-478f-81ba-7d2aca664f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:49.914829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469206d-4a38-478f-81ba-7d2aca664f54 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:49.915121 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469206d-4a38-478f-81ba-7d2aca664f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:49.915404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469206d-4a38-478f-81ba-7d2aca664f54 None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:49.921073 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469206d-4a38-478f-81ba-7d2aca664f54 None admin] RBAC: Authorizing `identity:get_project(project_id=d2415dcdc76d43c486190d67433cf01b)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:49.921299 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-95a48038-4244-4958-a38e-05d41c66e46f None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:49.922475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-95a48038-4244-4958-a38e-05d41c66e46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:49.922581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469206d-4a38-478f-81ba-7d2aca664f54 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:49.924736 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 132/268] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:49 2022] GET /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:49.925659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469206d-4a38-478f-81ba-7d2aca664f54 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:50.008696 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 137/269] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:20:50.008696 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a48038-4244-4958-a38e-05d41c66e46f None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:50.103018 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469206d-4a38-478f-81ba-7d2aca664f54 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:50.165510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:50.171935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a48038-4244-4958-a38e-05d41c66e46f None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:50.175740 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e51fc3-c4bc-46e2-aa45-89ba4ce98058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:50.176419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e51fc3-c4bc-46e2-aa45-89ba4ce98058 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:50.176707 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e51fc3-c4bc-46e2-aa45-89ba4ce98058 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:50.176989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e51fc3-c4bc-46e2-aa45-89ba4ce98058 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:50.177868 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e51fc3-c4bc-46e2-aa45-89ba4ce98058 None admin] RBAC: Adding query filter params (role.id=1423a24997e840608b6c522958481fde, scope.project.id=d2415dcdc76d43c486190d67433cf01b, user.id=fe5a62f54a5d46bf911f78bddcffb61f) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:50.178186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e51fc3-c4bc-46e2-aa45-89ba4ce98058 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:50.179646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e51fc3-c4bc-46e2-aa45-89ba4ce98058 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:50.196081 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 133/270] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:20:49 2022] GET /identity/v3/role_assignments?user.id=fe5a62f54a5d46bf911f78bddcffb61f&scope.project.id=d2415dcdc76d43c486190d67433cf01b&role.id=1423a24997e840608b6c522958481fde => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:50.196723 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e51fc3-c4bc-46e2-aa45-89ba4ce98058 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:50.254009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323df222-b855-49b4-a12f-c0fc069f192e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:50.254705 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323df222-b855-49b4-a12f-c0fc069f192e None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:50.254995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323df222-b855-49b4-a12f-c0fc069f192e None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:50.255281 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323df222-b855-49b4-a12f-c0fc069f192e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:50.256155 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323df222-b855-49b4-a12f-c0fc069f192e None admin] RBAC: Adding query filter params (role.id=25a16d8ea4084e9eacdebb7a4dcd2337, scope.project.id=d2415dcdc76d43c486190d67433cf01b, user.id=fe5a62f54a5d46bf911f78bddcffb61f) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:50.256476 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323df222-b855-49b4-a12f-c0fc069f192e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:50.258004 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323df222-b855-49b4-a12f-c0fc069f192e None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:50.265169 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 138/271] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:20:50 2022] GET /identity/v3/role_assignments?user.id=fe5a62f54a5d46bf911f78bddcffb61f&scope.project.id=d2415dcdc76d43c486190d67433cf01b&role.id=25a16d8ea4084e9eacdebb7a4dcd2337 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:50.265938 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:50.394617 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e51fc3-c4bc-46e2-aa45-89ba4ce98058 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:50.443117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163de30b-bae6-4ba6-a809-cb04c0b58b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:50.443965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163de30b-bae6-4ba6-a809-cb04c0b58b71 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:50.444256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163de30b-bae6-4ba6-a809-cb04c0b58b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:50.444540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163de30b-bae6-4ba6-a809-cb04c0b58b71 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:50.466144 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163de30b-bae6-4ba6-a809-cb04c0b58b71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:50.474531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-163de30b-bae6-4ba6-a809-cb04c0b58b71 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:50.474531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:50.474531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:50.474531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:50.474531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:50.474531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:50.474531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:50.474531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:50.474531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:50.474531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:50.474531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:50.474531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:50.474531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:50.474531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:50.474531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:50.474531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:50.475518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:50.475518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:50.475518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:50.475518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:50.475518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:50.475518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:50.475518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:50.475518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:50.475518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:50.475518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:50.475518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:50.475518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:50.475518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:50.475518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:50.475518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:50.475518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:50.476773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:50.476773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:50.476773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:50.476773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:50.476773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:50.476773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:50.476773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:50.476773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:50.476773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:50.476773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:50.476773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:50.476773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:20:50.476773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 134/272] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:50.482007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323df222-b855-49b4-a12f-c0fc069f192e None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:50.521600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9997e6-d6a7-42a1-9c54-af562d18ae24 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:50.522216 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9997e6-d6a7-42a1-9c54-af562d18ae24 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:50.522696 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9997e6-d6a7-42a1-9c54-af562d18ae24 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:50.525562 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 135/273] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:20:50.530048 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb7a44-2c4d-4080-a184-c54b91d5284e None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:50.530599 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb7a44-2c4d-4080-a184-c54b91d5284e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:50.531095 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb7a44-2c4d-4080-a184-c54b91d5284e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:50.550673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-50fb7a44-2c4d-4080-a184-c54b91d5284e None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:50.552236 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-50fb7a44-2c4d-4080-a184-c54b91d5284e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:50.632093 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 136/274] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:20:50.640743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323df222-b855-49b4-a12f-c0fc069f192e None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:50.657703 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb7a44-2c4d-4080-a184-c54b91d5284e None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:50.691849 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db11861-f021-40f1-ba45-4cd8bdc6cd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:50.692550 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db11861-f021-40f1-ba45-4cd8bdc6cd51 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:50.692845 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db11861-f021-40f1-ba45-4cd8bdc6cd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:50.693132 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db11861-f021-40f1-ba45-4cd8bdc6cd51 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:50.693957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db11861-f021-40f1-ba45-4cd8bdc6cd51 None admin] RBAC: Adding query filter params (name=member) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:50.694332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db11861-f021-40f1-ba45-4cd8bdc6cd51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:50.694586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:50.695736 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db11861-f021-40f1-ba45-4cd8bdc6cd51 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:50.708817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 139/275] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:20:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:50.714941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db11861-f021-40f1-ba45-4cd8bdc6cd51 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:50.777531 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:50.803682 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb7a44-2c4d-4080-a184-c54b91d5284e None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:50.847318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db11861-f021-40f1-ba45-4cd8bdc6cd51 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:50.848924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de51704-78f9-42c6-94a7-dea865884353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:50.849768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de51704-78f9-42c6-94a7-dea865884353 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:50.850060 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de51704-78f9-42c6-94a7-dea865884353 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:50.850342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de51704-78f9-42c6-94a7-dea865884353 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:50.851571 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de51704-78f9-42c6-94a7-dea865884353 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:50.853023 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de51704-78f9-42c6-94a7-dea865884353 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:50.854698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 137/276] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:20:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:50.855905 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060045b-962d-454a-86e9-2bb27ad2bb44 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:50.856208 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060045b-962d-454a-86e9-2bb27ad2bb44 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:50.856576 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060045b-962d-454a-86e9-2bb27ad2bb44 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:50.858078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 138/277] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:50.862697 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0060045b-962d-454a-86e9-2bb27ad2bb44 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:50.889678 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c6651-ad08-4f51-8786-7be978fd0ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:50.890843 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c6651-ad08-4f51-8786-7be978fd0ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:50.891327 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c6651-ad08-4f51-8786-7be978fd0ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:50.891622 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c6651-ad08-4f51-8786-7be978fd0ac3 None admin] PATH_INFO: `/v3/users/1830cc1b6a8443ff8b06356e19ab38ac` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:50.905823 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444c6651-ad08-4f51-8786-7be978fd0ac3 None admin] RBAC: Authorizing `identity:get_user(user_id=1830cc1b6a8443ff8b06356e19ab38ac)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:50.907333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444c6651-ad08-4f51-8786-7be978fd0ac3 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:50.909354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 140/278] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:50 2022] GET /identity/v3/users/1830cc1b6a8443ff8b06356e19ab38ac => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:50.910656 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac225d7-c106-42f5-8928-24f82d1e21be None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:50.911055 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac225d7-c106-42f5-8928-24f82d1e21be None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:50.911351 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac225d7-c106-42f5-8928-24f82d1e21be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:50.928758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-0ac225d7-c106-42f5-8928-24f82d1e21be None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:50.929995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0ac225d7-c106-42f5-8928-24f82d1e21be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:50.964519 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:50.977069 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0060045b-962d-454a-86e9-2bb27ad2bb44 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:51.004530 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 141/279] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:51.005411 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac225d7-c106-42f5-8928-24f82d1e21be None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:51.024730 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf89dc-10a1-440b-ab42-0cb889186c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:51.025712 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf89dc-10a1-440b-ab42-0cb889186c8e None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:51.026164 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf89dc-10a1-440b-ab42-0cb889186c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:51.026598 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf89dc-10a1-440b-ab42-0cb889186c8e None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:51.027056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:51.028740 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf89dc-10a1-440b-ab42-0cb889186c8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:51.031275 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf89dc-10a1-440b-ab42-0cb889186c8e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:51.058116 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 139/280] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 17:20:50 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:20:51.060385 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf89dc-10a1-440b-ab42-0cb889186c8e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:51.147953 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac225d7-c106-42f5-8928-24f82d1e21be None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:51.185292 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddd787-c46d-4abb-bfaa-b7d249fdbce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:51.185894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddd787-c46d-4abb-bfaa-b7d249fdbce3 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:51.186212 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddd787-c46d-4abb-bfaa-b7d249fdbce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:51.186577 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddd787-c46d-4abb-bfaa-b7d249fdbce3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:51.187615 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ddd787-c46d-4abb-bfaa-b7d249fdbce3 None admin] RBAC: Adding query filter params (role.id=483146fdc4ac47cf97c04a9f66c091b4, user.id=1830cc1b6a8443ff8b06356e19ab38ac, scope.system=all) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:51.187846 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ddd787-c46d-4abb-bfaa-b7d249fdbce3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:51.189937 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ddd787-c46d-4abb-bfaa-b7d249fdbce3 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:51.198124 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 142/281] 10.222.0.40 () {56 vars in 1308 bytes} [Thu Jun 16 17:20:51 2022] GET /identity/v3/role_assignments?user.id=1830cc1b6a8443ff8b06356e19ab38ac&scope.system=all&role.id=483146fdc4ac47cf97c04a9f66c091b4 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:51.199558 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896e00d-4e9e-40fe-a5de-b15b2a71bae8 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:51.199853 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896e00d-4e9e-40fe-a5de-b15b2a71bae8 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:51.200140 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896e00d-4e9e-40fe-a5de-b15b2a71bae8 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:51.201075 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf89dc-10a1-440b-ab42-0cb889186c8e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:51.206671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 143/282] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:51.209353 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd218bb3-3caf-4b49-a6c7-612c9a411f64 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:51.209657 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd218bb3-3caf-4b49-a6c7-612c9a411f64 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:51.209926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd218bb3-3caf-4b49-a6c7-612c9a411f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:51.227841 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-cd218bb3-3caf-4b49-a6c7-612c9a411f64 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:51.229055 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-cd218bb3-3caf-4b49-a6c7-612c9a411f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:51.270013 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fb1a7-f120-475c-878f-e248583f0f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:51.270911 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fb1a7-f120-475c-878f-e248583f0f1b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:51.271385 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fb1a7-f120-475c-878f-e248583f0f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:51.271686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fb1a7-f120-475c-878f-e248583f0f1b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:51.292588 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375fb1a7-f120-475c-878f-e248583f0f1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:51.302342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-375fb1a7-f120-475c-878f-e248583f0f1b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:20:51.302342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:51.302342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:51.302342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:51.302342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:51.302342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:51.302342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:51.302342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:51.302342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:51.302342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:51.302342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:51.302342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:51.302342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:51.302342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:51.302342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:51.302342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:51.303333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:51.303333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:51.303333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:51.303333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:51.303333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:51.303333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:51.303333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:51.303333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:51.303333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:51.303333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:51.303333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:51.303333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:51.303333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:51.303333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:51.303333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:51.303333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:51.304183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:51.304183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:51.304183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:51.304183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:51.304183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:51.304183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:51.304183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:51.304183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:51.304183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:51.304183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:51.304183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:20:51.304183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:20:51.307110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 140/283] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:51.313460 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fb1a7-f120-475c-878f-e248583f0f1b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:51.327521 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:51.334917 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 144/284] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:51.364394 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd218bb3-3caf-4b49-a6c7-612c9a411f64 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:51.372712 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:51.450619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fb1a7-f120-475c-878f-e248583f0f1b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:51.500550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809e39b-5a72-4d3c-9414-130504c31cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:51.501364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809e39b-5a72-4d3c-9414-130504c31cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:51.501720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809e39b-5a72-4d3c-9414-130504c31cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:51.502131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809e39b-5a72-4d3c-9414-130504c31cc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:51.502982 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6809e39b-5a72-4d3c-9414-130504c31cc8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:51.503360 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6809e39b-5a72-4d3c-9414-130504c31cc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:51.504823 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6809e39b-5a72-4d3c-9414-130504c31cc8 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:51.505990 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd218bb3-3caf-4b49-a6c7-612c9a411f64 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:51.522046 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 141/285] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:20:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:51.527199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809e39b-5a72-4d3c-9414-130504c31cc8 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:51.577925 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fed84-1252-448b-bbe4-f000f96cf5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:51.578735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fed84-1252-448b-bbe4-f000f96cf5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:51.579018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fed84-1252-448b-bbe4-f000f96cf5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:51.579298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fed84-1252-448b-bbe4-f000f96cf5d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:51.606195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1fed84-1252-448b-bbe4-f000f96cf5d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:51.621181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-ea1fed84-1252-448b-bbe4-f000f96cf5d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:51.621181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:51.621181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:51.621181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:51.621181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:51.621181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:51.621181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:51.621181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:51.621181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:51.621181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:51.621181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:51.621181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:51.621181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:51.621181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:51.621181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:51.621181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:51.622162 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:51.622162 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:51.622162 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:51.622162 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:51.622162 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:51.622162 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:51.622162 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:51.622162 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:51.622162 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:51.622162 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:51.622162 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:51.622162 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:51.622162 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:51.622162 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:51.622162 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:51.622162 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:51.622982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:51.622982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:51.622982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:51.622982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:51.622982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:51.622982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:51.622982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:51.622982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:51.622982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:51.622982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:51.622982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:51.622982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:20:51.625978 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 145/286] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:20:51.631337 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:51.631883 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fed84-1252-448b-bbe4-f000f96cf5d4 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:51.690821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809e39b-5a72-4d3c-9414-130504c31cc8 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:51.737944 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7f51a-16f7-46b6-a52b-65bd1dd3f1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:51.739130 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7f51a-16f7-46b6-a52b-65bd1dd3f1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:51.739598 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7f51a-16f7-46b6-a52b-65bd1dd3f1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:51.740057 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7f51a-16f7-46b6-a52b-65bd1dd3f1f2 None admin] PATH_INFO: `/v3/users/5ad4c71102db46dfa8af097faf29ecf2` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:51.741490 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7f51a-16f7-46b6-a52b-65bd1dd3f1f2 None admin] RBAC: Authorizing `identity:get_user(user_id=5ad4c71102db46dfa8af097faf29ecf2)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:51.743251 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7f51a-16f7-46b6-a52b-65bd1dd3f1f2 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:51.745419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 142/287] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:51 2022] GET /identity/v3/users/5ad4c71102db46dfa8af097faf29ecf2 => generated 297 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:51.751149 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7f51a-16f7-46b6-a52b-65bd1dd3f1f2 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:51.770649 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:51.814125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fed84-1252-448b-bbe4-f000f96cf5d4 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:51.861580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc5ce0-1290-479e-b55a-3c501ba2af88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:51.863278 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc5ce0-1290-479e-b55a-3c501ba2af88 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:51.863580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc5ce0-1290-479e-b55a-3c501ba2af88 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:51.863866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc5ce0-1290-479e-b55a-3c501ba2af88 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:51.864639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc5ce0-1290-479e-b55a-3c501ba2af88 None admin] RBAC: Adding query filter params (name=member) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:51.864958 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc5ce0-1290-479e-b55a-3c501ba2af88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:51.866740 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc5ce0-1290-479e-b55a-3c501ba2af88 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:51.877519 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 146/288] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:20:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:51.884180 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc5ce0-1290-479e-b55a-3c501ba2af88 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:51.991309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7f51a-16f7-46b6-a52b-65bd1dd3f1f2 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:52.038434 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc5ce0-1290-479e-b55a-3c501ba2af88 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:52.058566 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb7314-4339-4169-8343-44b91bc2e725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:52.059451 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb7314-4339-4169-8343-44b91bc2e725 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:52.059738 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb7314-4339-4169-8343-44b91bc2e725 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:52.060016 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb7314-4339-4169-8343-44b91bc2e725 None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:52.061309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb7314-4339-4169-8343-44b91bc2e725 None admin] RBAC: Authorizing `identity:get_project(project_id=d2415dcdc76d43c486190d67433cf01b)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:52.062786 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb7314-4339-4169-8343-44b91bc2e725 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:52.065054 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 143/289] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:51 2022] GET /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b => generated 302 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:52.073524 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb7314-4339-4169-8343-44b91bc2e725 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:52.079759 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:52.089570 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:52.110232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a287c3-40b3-4cca-8d11-9cdaa73d08b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:52.112462 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a287c3-40b3-4cca-8d11-9cdaa73d08b6 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:52.113202 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a287c3-40b3-4cca-8d11-9cdaa73d08b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:52.115223 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a287c3-40b3-4cca-8d11-9cdaa73d08b6 None admin] PATH_INFO: `/v3/users/b3b50699be764e628210e956b938cc0a` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:52.117678 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a287c3-40b3-4cca-8d11-9cdaa73d08b6 None admin] RBAC: Authorizing `identity:get_user(user_id=b3b50699be764e628210e956b938cc0a)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:52.120916 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a287c3-40b3-4cca-8d11-9cdaa73d08b6 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:52.123666 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 147/290] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:51 2022] GET /identity/v3/users/b3b50699be764e628210e956b938cc0a => generated 297 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:52.131689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a287c3-40b3-4cca-8d11-9cdaa73d08b6 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:52.278391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb7314-4339-4169-8343-44b91bc2e725 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:52.317416 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a287c3-40b3-4cca-8d11-9cdaa73d08b6 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:52.340914 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d937be-1c2e-421e-91cc-19ea4f7d74f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:52.341854 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d937be-1c2e-421e-91cc-19ea4f7d74f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:52.342144 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d937be-1c2e-421e-91cc-19ea4f7d74f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:52.342427 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d937be-1c2e-421e-91cc-19ea4f7d74f5 None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b/users/5ad4c71102db46dfa8af097faf29ecf2/roles/075fc2a9c4184b9db26b044f3d963233` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:52.345396 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d937be-1c2e-421e-91cc-19ea4f7d74f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d2415dcdc76d43c486190d67433cf01b, user_id=5ad4c71102db46dfa8af097faf29ecf2, role_id=075fc2a9c4184b9db26b044f3d963233)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:52.347583 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d937be-1c2e-421e-91cc-19ea4f7d74f5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:52.371494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 144/291] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:20:52 2022] PUT /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b/users/5ad4c71102db46dfa8af097faf29ecf2/roles/075fc2a9c4184b9db26b044f3d963233 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:20:52.381717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24e684-2934-4b55-8fd9-06a4d0412b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:52.382439 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24e684-2934-4b55-8fd9-06a4d0412b69 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:52.382735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24e684-2934-4b55-8fd9-06a4d0412b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:52.383026 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24e684-2934-4b55-8fd9-06a4d0412b69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:52.464719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:52.473638 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24e684-2934-4b55-8fd9-06a4d0412b69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:52.475133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24e684-2934-4b55-8fd9-06a4d0412b69 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:52.619565 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 148/292] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:20:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:20:52.625127 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d937be-1c2e-421e-91cc-19ea4f7d74f5 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:52.700798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:52.746040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d937be-1c2e-421e-91cc-19ea4f7d74f5 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:52.784335 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b224e-685f-4e07-96ab-a454d179b4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:52.785303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b224e-685f-4e07-96ab-a454d179b4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:52.785596 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b224e-685f-4e07-96ab-a454d179b4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:52.785878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b224e-685f-4e07-96ab-a454d179b4a0 None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:52.787094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9b224e-685f-4e07-96ab-a454d179b4a0 None admin] RBAC: Authorizing `identity:get_project(project_id=d2415dcdc76d43c486190d67433cf01b)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:52.788543 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9b224e-685f-4e07-96ab-a454d179b4a0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:52.790587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 145/293] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:52 2022] GET /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:52.796295 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24e684-2934-4b55-8fd9-06a4d0412b69 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:52.925912 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:52.933122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24e684-2934-4b55-8fd9-06a4d0412b69 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:52.980109 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1dd617-1428-49ab-987a-f5cba47daceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:52.980786 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1dd617-1428-49ab-987a-f5cba47daceb None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:52.981079 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1dd617-1428-49ab-987a-f5cba47daceb None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:52.981399 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1dd617-1428-49ab-987a-f5cba47daceb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:52.982253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1dd617-1428-49ab-987a-f5cba47daceb None admin] RBAC: Adding query filter params (role.id=483146fdc4ac47cf97c04a9f66c091b4, scope.project.id=d2415dcdc76d43c486190d67433cf01b, user.id=b3b50699be764e628210e956b938cc0a) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:52.982576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1dd617-1428-49ab-987a-f5cba47daceb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:52.984028 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1dd617-1428-49ab-987a-f5cba47daceb None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:52.994088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 149/294] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:20:52 2022] GET /identity/v3/role_assignments?user.id=b3b50699be764e628210e956b938cc0a&scope.project.id=d2415dcdc76d43c486190d67433cf01b&role.id=483146fdc4ac47cf97c04a9f66c091b4 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:53.779827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa0ebc-aae8-49bb-946d-7e360aa3e534 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:53.780351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa0ebc-aae8-49bb-946d-7e360aa3e534 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:53.780491 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa0ebc-aae8-49bb-946d-7e360aa3e534 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:53.782364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 146/295] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:53.788110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fe3b6-f252-442b-a97a-93ca67edfafd None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:53.788293 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b83f79-2483-4b37-82ee-bc3f2189f9b8 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:53.790097 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b83f79-2483-4b37-82ee-bc3f2189f9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:53.790223 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fe3b6-f252-442b-a97a-93ca67edfafd None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:53.790316 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b83f79-2483-4b37-82ee-bc3f2189f9b8 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:53.790401 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fe3b6-f252-442b-a97a-93ca67edfafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:53.790749 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 147/296] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:20:53.796068 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730aa8ce-80d4-4b9f-9e83-b928241151fe None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:53.796597 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730aa8ce-80d4-4b9f-9e83-b928241151fe None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:53.797074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730aa8ce-80d4-4b9f-9e83-b928241151fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:53.812488 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-b43fe3b6-f252-442b-a97a-93ca67edfafd None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:53.813722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b43fe3b6-f252-442b-a97a-93ca67edfafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:53.814996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-730aa8ce-80d4-4b9f-9e83-b928241151fe None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:53.816213 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-730aa8ce-80d4-4b9f-9e83-b928241151fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:53.890650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 148/297] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:20:53.892173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adade4-6e9c-4da6-9562-911a5503d96e None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:53.892755 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adade4-6e9c-4da6-9562-911a5503d96e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:53.893259 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adade4-6e9c-4da6-9562-911a5503d96e None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:53.895123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 149/298] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:53.897509 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 150/299] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:20:53.906575 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a31bae-5b19-4b2a-a023-69e82edb1f46 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:53.906861 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a31bae-5b19-4b2a-a023-69e82edb1f46 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:53.907146 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a31bae-5b19-4b2a-a023-69e82edb1f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:53.915854 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fe3b6-f252-442b-a97a-93ca67edfafd None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:53.925094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-c6a31bae-5b19-4b2a-a023-69e82edb1f46 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:53.926312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c6a31bae-5b19-4b2a-a023-69e82edb1f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:53.989505 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:54.003011 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 150/300] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:54.004328 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a31bae-5b19-4b2a-a023-69e82edb1f46 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:54.015849 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:54.026957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fe3b6-f252-442b-a97a-93ca67edfafd None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:54.068620 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9ab68-4167-46d4-af96-0e082d559ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:54.069359 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9ab68-4167-46d4-af96-0e082d559ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:54.069762 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9ab68-4167-46d4-af96-0e082d559ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:54.070039 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9ab68-4167-46d4-af96-0e082d559ce5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:54.081524 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9ab68-4167-46d4-af96-0e082d559ce5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:54.089749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-78f9ab68-4167-46d4-af96-0e082d559ce5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:20:54.089749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:54.089749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:54.089749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:54.089749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:54.089749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:54.089749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:54.089749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:54.089749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:54.089749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:54.089749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:54.089749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:54.089749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:54.089749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:54.089749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:54.089749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:54.090675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:54.090675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:54.090675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:54.090675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:54.090675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:54.090675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:54.090675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:54.090675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:54.090675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:54.090675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:54.090675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:54.090675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:54.090675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:54.090675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:54.090675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:54.090675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:54.091493 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:54.091493 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:54.091493 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:54.091493 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:54.091493 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:54.091493 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:54.091493 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:54.091493 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:54.091493 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:54.091493 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:54.091493 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:20:54.091493 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:20:54.091493 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 151/301] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:54.094029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9ab68-4167-46d4-af96-0e082d559ce5 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:54.138162 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a31bae-5b19-4b2a-a023-69e82edb1f46 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:54.175336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235b874-e646-4af1-ba0c-bcc9aeb6f69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:54.175843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235b874-e646-4af1-ba0c-bcc9aeb6f69e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:54.176137 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235b874-e646-4af1-ba0c-bcc9aeb6f69e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:54.176419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235b874-e646-4af1-ba0c-bcc9aeb6f69e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:54.177620 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235b874-e646-4af1-ba0c-bcc9aeb6f69e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:54.179066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235b874-e646-4af1-ba0c-bcc9aeb6f69e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:54.180694 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 151/302] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:20:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:54.181513 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235b874-e646-4af1-ba0c-bcc9aeb6f69e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:54.205092 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9ab68-4167-46d4-af96-0e082d559ce5 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:54.241907 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc300f-ee8b-4eca-b37c-f64553f3b6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:54.242720 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc300f-ee8b-4eca-b37c-f64553f3b6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:54.243438 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc300f-ee8b-4eca-b37c-f64553f3b6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:54.243733 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc300f-ee8b-4eca-b37c-f64553f3b6c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:54.250096 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:54.254812 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc300f-ee8b-4eca-b37c-f64553f3b6c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:54.263467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-28bc300f-ee8b-4eca-b37c-f64553f3b6c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:54.263467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:54.263467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:54.263467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:54.263467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:54.263467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:54.263467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:54.263467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:54.263467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:54.263467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:54.263467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:54.263467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:54.263467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:54.263467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:54.263467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:54.263467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:54.264406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:54.264406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:54.264406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:54.264406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:54.264406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:54.264406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:54.264406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:54.264406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:54.264406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:54.264406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:54.264406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:54.264406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:54.264406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:54.264406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:54.264406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:54.264406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:54.265300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:54.265300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:54.265300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:54.265300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:54.265300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:54.265300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:54.265300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:54.265300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:54.265300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:54.265300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:54.265300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:54.265300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:20:54.265300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 152/303] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:54.265300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc300f-ee8b-4eca-b37c-f64553f3b6c1 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:54.279168 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:54.303823 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235b874-e646-4af1-ba0c-bcc9aeb6f69e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:54.348597 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c2a4e-de8a-437c-a639-8387bb29ee92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:54.349228 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c2a4e-de8a-437c-a639-8387bb29ee92 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:54.349521 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c2a4e-de8a-437c-a639-8387bb29ee92 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:54.349806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c2a4e-de8a-437c-a639-8387bb29ee92 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:54.350563 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c2a4e-de8a-437c-a639-8387bb29ee92 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:54.350880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c2a4e-de8a-437c-a639-8387bb29ee92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:54.352273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c2a4e-de8a-437c-a639-8387bb29ee92 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:54.360796 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 152/304] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:20:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:54.360796 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c2a4e-de8a-437c-a639-8387bb29ee92 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:54.389429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc300f-ee8b-4eca-b37c-f64553f3b6c1 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:54.429873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d8c33-3746-4048-aac2-e49374bc69bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:54.430841 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d8c33-3746-4048-aac2-e49374bc69bb None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:54.431318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d8c33-3746-4048-aac2-e49374bc69bb None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:54.431671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d8c33-3746-4048-aac2-e49374bc69bb None admin] PATH_INFO: `/v3/groups` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:54.432371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d8c33-3746-4048-aac2-e49374bc69bb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:54.433845 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d8c33-3746-4048-aac2-e49374bc69bb None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:54.461186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 153/305] 10.222.0.40 () {60 vars in 1134 bytes} [Thu Jun 16 17:20:54 2022] POST /identity/v3/groups => generated 229 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:20:54.461186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d8c33-3746-4048-aac2-e49374bc69bb None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:54.480852 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c2a4e-de8a-437c-a639-8387bb29ee92 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:54.509836 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:54.525295 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a29a49-e0e4-428a-9148-144693c42f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:54.526168 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a29a49-e0e4-428a-9148-144693c42f14 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:54.526589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a29a49-e0e4-428a-9148-144693c42f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:54.526992 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a29a49-e0e4-428a-9148-144693c42f14 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:54.527960 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a29a49-e0e4-428a-9148-144693c42f14 None admin] RBAC: Adding query filter params (name=member) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:54.528373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a29a49-e0e4-428a-9148-144693c42f14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:54.530017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a29a49-e0e4-428a-9148-144693c42f14 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:54.540442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 153/306] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:20:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:54.546217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a29a49-e0e4-428a-9148-144693c42f14 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:54.553171 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:54.607808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d8c33-3746-4048-aac2-e49374bc69bb None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:54.651745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4552b7f-67b1-4738-b84e-2849602bfc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:54.652642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4552b7f-67b1-4738-b84e-2849602bfc6b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:54.652981 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4552b7f-67b1-4738-b84e-2849602bfc6b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:54.653255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4552b7f-67b1-4738-b84e-2849602bfc6b None admin] PATH_INFO: `/v3/users/914c71698f894d7cbf0bd6f04b96f0ff` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:54.670834 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4552b7f-67b1-4738-b84e-2849602bfc6b None admin] RBAC: Authorizing `identity:get_user(user_id=914c71698f894d7cbf0bd6f04b96f0ff)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:54.673484 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4552b7f-67b1-4738-b84e-2849602bfc6b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:54.675995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 154/307] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:54 2022] GET /identity/v3/users/914c71698f894d7cbf0bd6f04b96f0ff => generated 293 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:54.682063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4552b7f-67b1-4738-b84e-2849602bfc6b None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:54.687534 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a29a49-e0e4-428a-9148-144693c42f14 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:54.725609 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f41f8-7ff2-42cc-a53a-f0414c641e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:54.726509 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f41f8-7ff2-42cc-a53a-f0414c641e42 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:54.726799 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f41f8-7ff2-42cc-a53a-f0414c641e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:54.727084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f41f8-7ff2-42cc-a53a-f0414c641e42 None admin] PATH_INFO: `/v3/users/1830cc1b6a8443ff8b06356e19ab38ac` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:54.728314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f41f8-7ff2-42cc-a53a-f0414c641e42 None admin] RBAC: Authorizing `identity:get_user(user_id=1830cc1b6a8443ff8b06356e19ab38ac)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:54.729803 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f41f8-7ff2-42cc-a53a-f0414c641e42 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:54.731769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 154/308] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:54 2022] GET /identity/v3/users/1830cc1b6a8443ff8b06356e19ab38ac => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:54.738251 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f41f8-7ff2-42cc-a53a-f0414c641e42 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:54.810062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:54.821438 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4552b7f-67b1-4738-b84e-2849602bfc6b None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:54.863538 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:54.877368 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da96359-8b65-415f-aafc-10a6873f739e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:54.878366 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da96359-8b65-415f-aafc-10a6873f739e None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:54.878805 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da96359-8b65-415f-aafc-10a6873f739e None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:54.879233 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da96359-8b65-415f-aafc-10a6873f739e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:54.879523 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f41f8-7ff2-42cc-a53a-f0414c641e42 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:54.880447 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da96359-8b65-415f-aafc-10a6873f739e None admin] RBAC: Adding query filter params (role.id=075fc2a9c4184b9db26b044f3d963233, user.id=914c71698f894d7cbf0bd6f04b96f0ff, scope.system=all) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:54.880926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da96359-8b65-415f-aafc-10a6873f739e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:54.883874 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da96359-8b65-415f-aafc-10a6873f739e None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:54.897704 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 155/309] 10.222.0.40 () {56 vars in 1308 bytes} [Thu Jun 16 17:20:54 2022] GET /identity/v3/role_assignments?user.id=914c71698f894d7cbf0bd6f04b96f0ff&scope.system=all&role.id=075fc2a9c4184b9db26b044f3d963233 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:54.903720 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0250f-34bf-4463-9420-7d96346400da None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:54.904014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0250f-34bf-4463-9420-7d96346400da None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:54.904283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0250f-34bf-4463-9420-7d96346400da None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:54.905972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 156/310] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:20:54.912199 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2b816-5427-476a-b368-188fcfc4b69f None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:54.912504 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2b816-5427-476a-b368-188fcfc4b69f None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:54.912775 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2b816-5427-476a-b368-188fcfc4b69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:54.928390 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc0e8c-e092-443c-ad51-7797e79ce661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:54.929319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc0e8c-e092-443c-ad51-7797e79ce661 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:54.929608 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc0e8c-e092-443c-ad51-7797e79ce661 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:54.929889 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc0e8c-e092-443c-ad51-7797e79ce661 None admin] PATH_INFO: `/v3/system/users/1830cc1b6a8443ff8b06356e19ab38ac/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:54.930782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-e5d2b816-5427-476a-b368-188fcfc4b69f None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:54.931436 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc0e8c-e092-443c-ad51-7797e79ce661 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1830cc1b6a8443ff8b06356e19ab38ac, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:54.932022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-e5d2b816-5427-476a-b368-188fcfc4b69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:54.932900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc0e8c-e092-443c-ad51-7797e79ce661 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:54.946439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 155/311] 10.222.0.40 () {58 vars in 1334 bytes} [Thu Jun 16 17:20:54 2022] PUT /identity/v3/system/users/1830cc1b6a8443ff8b06356e19ab38ac/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:20:55.023796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 157/312] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:20:55.050547 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc0e8c-e092-443c-ad51-7797e79ce661 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:55.176154 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc0e8c-e092-443c-ad51-7797e79ce661 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:55.193428 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:55.213117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16705f7-68e2-4ca9-b74f-2b96d65a0851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:55.213959 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16705f7-68e2-4ca9-b74f-2b96d65a0851 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:55.214251 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16705f7-68e2-4ca9-b74f-2b96d65a0851 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:55.214535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16705f7-68e2-4ca9-b74f-2b96d65a0851 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:55.225914 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16705f7-68e2-4ca9-b74f-2b96d65a0851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:55.234314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-a16705f7-68e2-4ca9-b74f-2b96d65a0851 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:20:55.234314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:55.234314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:55.234314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:55.234314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:55.234314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:55.234314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:55.234314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:55.234314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:55.234314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:55.234314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:55.234314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:55.234314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:55.234314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:55.234314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:55.234314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:55.235253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:55.235253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:55.235253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:55.235253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:55.235253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:55.235253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:55.235253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:55.235253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:55.235253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:55.235253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:55.235253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:55.235253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:55.235253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:55.235253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:55.235253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:55.235253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:55.236633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:55.236633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:55.236633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:55.236633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:55.236633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:55.236633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:55.236633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:55.236633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:55.236633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:55.236633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:55.236633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:20:55.236633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:20:55.236633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 156/313] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:20:55.245872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2b816-5427-476a-b368-188fcfc4b69f None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:55.354599 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2b816-5427-476a-b368-188fcfc4b69f None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:55.378500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:55.391124 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d69ea4-c0f7-42a8-a7f9-ce60657ae797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:55.391803 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d69ea4-c0f7-42a8-a7f9-ce60657ae797 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:55.392090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d69ea4-c0f7-42a8-a7f9-ce60657ae797 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:55.392900 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d69ea4-c0f7-42a8-a7f9-ce60657ae797 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:55.393698 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d69ea4-c0f7-42a8-a7f9-ce60657ae797 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:55.394001 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d69ea4-c0f7-42a8-a7f9-ce60657ae797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:55.395401 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d69ea4-c0f7-42a8-a7f9-ce60657ae797 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:55.404963 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 158/314] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:20:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:55.411128 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16705f7-68e2-4ca9-b74f-2b96d65a0851 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:55.515930 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16705f7-68e2-4ca9-b74f-2b96d65a0851 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:55.550806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1712b5d8-6681-41e4-8fed-39867a66bd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:55.551701 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712b5d8-6681-41e4-8fed-39867a66bd11 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:55.551988 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712b5d8-6681-41e4-8fed-39867a66bd11 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:55.552268 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712b5d8-6681-41e4-8fed-39867a66bd11 None admin] PATH_INFO: `/v3/users/5ad4c71102db46dfa8af097faf29ecf2` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:55.553508 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1712b5d8-6681-41e4-8fed-39867a66bd11 None admin] RBAC: Authorizing `identity:get_user(user_id=5ad4c71102db46dfa8af097faf29ecf2)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:55.554974 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1712b5d8-6681-41e4-8fed-39867a66bd11 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:55.556617 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 157/315] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:55 2022] GET /identity/v3/users/5ad4c71102db46dfa8af097faf29ecf2 => generated 297 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:55.562086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d69ea4-c0f7-42a8-a7f9-ce60657ae797 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:55.669188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d69ea4-c0f7-42a8-a7f9-ce60657ae797 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:55.706301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9997b94b-c433-41fb-9bee-aeaa857ca110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:55.706987 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997b94b-c433-41fb-9bee-aeaa857ca110 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:55.707276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997b94b-c433-41fb-9bee-aeaa857ca110 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:55.707564 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997b94b-c433-41fb-9bee-aeaa857ca110 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:55.762267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:55.778901 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9997b94b-c433-41fb-9bee-aeaa857ca110 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:55.780361 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9997b94b-c433-41fb-9bee-aeaa857ca110 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:55.888291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 159/316] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:20:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:20:55.893904 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1712b5d8-6681-41e4-8fed-39867a66bd11 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:55.942757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:55.999391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1712b5d8-6681-41e4-8fed-39867a66bd11 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:56.035882 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd4fe4-d898-4b4f-8940-ccb095cdd373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:56.037080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd4fe4-d898-4b4f-8940-ccb095cdd373 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:56.037607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd4fe4-d898-4b4f-8940-ccb095cdd373 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:56.038108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd4fe4-d898-4b4f-8940-ccb095cdd373 None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:56.039514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd4fe4-d898-4b4f-8940-ccb095cdd373 None admin] RBAC: Authorizing `identity:get_project(project_id=d2415dcdc76d43c486190d67433cf01b)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:56.041195 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd4fe4-d898-4b4f-8940-ccb095cdd373 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:56.043121 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 158/317] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:20:55 2022] GET /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:56.049189 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9997b94b-c433-41fb-9bee-aeaa857ca110 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:56.144581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:56.161692 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9997b94b-c433-41fb-9bee-aeaa857ca110 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:56.197199 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2525245-eb17-4332-a846-775c7dd8b597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:56.197925 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2525245-eb17-4332-a846-775c7dd8b597 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:56.198213 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2525245-eb17-4332-a846-775c7dd8b597 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:56.198538 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2525245-eb17-4332-a846-775c7dd8b597 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:56.199385 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2525245-eb17-4332-a846-775c7dd8b597 None admin] RBAC: Adding query filter params (role.id=075fc2a9c4184b9db26b044f3d963233, scope.project.id=d2415dcdc76d43c486190d67433cf01b, user.id=5ad4c71102db46dfa8af097faf29ecf2) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:56.199702 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2525245-eb17-4332-a846-775c7dd8b597 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:56.201141 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2525245-eb17-4332-a846-775c7dd8b597 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:56.208347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 160/318] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:20:56 2022] GET /identity/v3/role_assignments?user.id=5ad4c71102db46dfa8af097faf29ecf2&scope.project.id=d2415dcdc76d43c486190d67433cf01b&role.id=075fc2a9c4184b9db26b044f3d963233 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:56.803427 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059abd6-a62b-484e-ba08-c73404945e81 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:56.803802 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059abd6-a62b-484e-ba08-c73404945e81 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:56.803989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059abd6-a62b-484e-ba08-c73404945e81 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:56.805462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 159/319] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:56.810571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a094e-10f1-4190-bcfb-c8abe674166e None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:56.811092 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a094e-10f1-4190-bcfb-c8abe674166e None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:56.811453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a094e-10f1-4190-bcfb-c8abe674166e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:56.830783 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-7f9a094e-10f1-4190-bcfb-c8abe674166e None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:56.832467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-7f9a094e-10f1-4190-bcfb-c8abe674166e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:56.909199 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 161/320] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:56.928892 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059abd6-a62b-484e-ba08-c73404945e81 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:57.033712 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059abd6-a62b-484e-ba08-c73404945e81 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:57.048162 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:57.079210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa2769-6608-49e4-bc53-6d86a97614f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:57.080535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa2769-6608-49e4-bc53-6d86a97614f7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:57.080914 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa2769-6608-49e4-bc53-6d86a97614f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:57.081099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa2769-6608-49e4-bc53-6d86a97614f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:57.082521 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa2769-6608-49e4-bc53-6d86a97614f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:57.083967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa2769-6608-49e4-bc53-6d86a97614f7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:57.089206 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 160/321] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:20:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:57.091614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a094e-10f1-4190-bcfb-c8abe674166e None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:57.181771 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:57.196208 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b6093-1455-436f-87cb-0c2f4edfdf83 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:57.196506 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b6093-1455-436f-87cb-0c2f4edfdf83 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:57.196815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b6093-1455-436f-87cb-0c2f4edfdf83 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:57.198330 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 161/322] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:20:57.203467 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577240a-785a-4624-8291-7bdf7d82a250 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:57.203753 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577240a-785a-4624-8291-7bdf7d82a250 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:57.204036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577240a-785a-4624-8291-7bdf7d82a250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:57.209230 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a094e-10f1-4190-bcfb-c8abe674166e None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:57.223995 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-5577240a-785a-4624-8291-7bdf7d82a250 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:57.225223 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5577240a-785a-4624-8291-7bdf7d82a250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:57.246728 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9c3ce-3756-4079-9096-5f401da6ed1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:57.247399 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9c3ce-3756-4079-9096-5f401da6ed1b None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:57.247685 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9c3ce-3756-4079-9096-5f401da6ed1b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:57.247964 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9c3ce-3756-4079-9096-5f401da6ed1b None admin] PATH_INFO: `/v3/groups` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:57.248650 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9c3ce-3756-4079-9096-5f401da6ed1b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:57.250184 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9c3ce-3756-4079-9096-5f401da6ed1b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:57.263578 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 162/323] 10.222.0.40 () {60 vars in 1134 bytes} [Thu Jun 16 17:20:57 2022] POST /identity/v3/groups => generated 226 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:20:57.264958 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdad975-0a9f-45eb-869f-75d8f2a908a7 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:57.265296 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdad975-0a9f-45eb-869f-75d8f2a908a7 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:57.265589 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdad975-0a9f-45eb-869f-75d8f2a908a7 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:57.269301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 163/324] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:57.273893 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feef79a-62d4-40e6-befb-9c7a183df1c8 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:57.274630 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feef79a-62d4-40e6-befb-9c7a183df1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:57.274989 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feef79a-62d4-40e6-befb-9c7a183df1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:57.293298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-2feef79a-62d4-40e6-befb-9c7a183df1c8 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:57.294527 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-2feef79a-62d4-40e6-befb-9c7a183df1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:57.302127 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 162/325] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:57.326509 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577240a-785a-4624-8291-7bdf7d82a250 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:57.380960 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 164/326] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:20:57.404950 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feef79a-62d4-40e6-befb-9c7a183df1c8 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:57.453607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577240a-785a-4624-8291-7bdf7d82a250 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:57.465490 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:57.488796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:57.491930 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9753bc-5fac-41b9-a457-ea8fb326ca50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:57.493409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9753bc-5fac-41b9-a457-ea8fb326ca50 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:57.493409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9753bc-5fac-41b9-a457-ea8fb326ca50 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:57.493409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9753bc-5fac-41b9-a457-ea8fb326ca50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:57.509088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9753bc-5fac-41b9-a457-ea8fb326ca50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:57.517602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-3b9753bc-5fac-41b9-a457-ea8fb326ca50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:57.517602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:57.517602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:57.517602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:57.517602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:57.517602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:57.517602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:57.517602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:57.517602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:57.517602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:57.517602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:57.517602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:57.517602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:57.517602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:57.517602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:57.517602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:57.518527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:57.518527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:57.518527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:57.518527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:57.518527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:57.518527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:57.518527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:57.518527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:57.518527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:57.518527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:57.518527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:57.518527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:57.518527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:57.518527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:57.518527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:57.518527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:57.519361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:57.519361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:57.519361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:57.519361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:57.519361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:57.519361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:57.519361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:57.519361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:57.519361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:57.519361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:57.519361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:20:57.519361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:20:57.519361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 163/327] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:20:57.529296 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9753bc-5fac-41b9-a457-ea8fb326ca50 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:57.535520 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feef79a-62d4-40e6-befb-9c7a183df1c8 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:57.582449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b6488-59cc-4dd0-a9df-5213b9e6f348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:57.583266 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b6488-59cc-4dd0-a9df-5213b9e6f348 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:57.583561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b6488-59cc-4dd0-a9df-5213b9e6f348 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:57.583843 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b6488-59cc-4dd0-a9df-5213b9e6f348 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:57.597260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b6488-59cc-4dd0-a9df-5213b9e6f348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:20:57.606144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-023b6488-59cc-4dd0-a9df-5213b9e6f348 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:20:57.606144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:20:57.606144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:20:57.606144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:20:57.606144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:20:57.606144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:20:57.606144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:20:57.606144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:20:57.606144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:20:57.606144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:20:57.606144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:20:57.606144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:20:57.606144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:20:57.606144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:20:57.606144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:20:57.606144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:20:57.607096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:20:57.607096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:20:57.607096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:20:57.607096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:20:57.607096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:20:57.607096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:20:57.607096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:20:57.607096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:20:57.607096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:20:57.607096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:20:57.607096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:20:57.607096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:20:57.607096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:20:57.607096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:20:57.607096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:20:57.607096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:20:57.607935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:20:57.607935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:20:57.607935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:20:57.607935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:20:57.607935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:20:57.607935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:20:57.607935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:20:57.607935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:20:57.607935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:20:57.607935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:20:57.607935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:20:57.607935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:20:57.607935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 165/328] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:20:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:20:57.614884 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b6488-59cc-4dd0-a9df-5213b9e6f348 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:57.668390 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9753bc-5fac-41b9-a457-ea8fb326ca50 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:57.708243 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86209a-246d-413b-879f-9afa07f780b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:57.709001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86209a-246d-413b-879f-9afa07f780b5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:57.709577 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86209a-246d-413b-879f-9afa07f780b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:57.709695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86209a-246d-413b-879f-9afa07f780b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:57.711091 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86209a-246d-413b-879f-9afa07f780b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:57.711408 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86209a-246d-413b-879f-9afa07f780b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:57.712963 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86209a-246d-413b-879f-9afa07f780b5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:57.725227 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 164/329] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:20:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:57.730497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86209a-246d-413b-879f-9afa07f780b5 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:57.768148 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b6488-59cc-4dd0-a9df-5213b9e6f348 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:57.773960 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:57.803764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:57.814693 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278da782-c48b-4be6-a3e1-b55b644ec6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:57.815796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278da782-c48b-4be6-a3e1-b55b644ec6ff None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:57.816313 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278da782-c48b-4be6-a3e1-b55b644ec6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:57.816818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278da782-c48b-4be6-a3e1-b55b644ec6ff None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:57.817872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278da782-c48b-4be6-a3e1-b55b644ec6ff None admin] RBAC: Adding query filter params (name=reader) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:57.818412 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278da782-c48b-4be6-a3e1-b55b644ec6ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:57.820123 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278da782-c48b-4be6-a3e1-b55b644ec6ff None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:57.829871 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 166/330] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:20:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:57.836623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278da782-c48b-4be6-a3e1-b55b644ec6ff None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:57.874638 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86209a-246d-413b-879f-9afa07f780b5 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:57.921583 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ef474-fb57-4843-a56a-9b0db57541a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:57.922402 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ef474-fb57-4843-a56a-9b0db57541a9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:57.922748 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ef474-fb57-4843-a56a-9b0db57541a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:57.923027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ef474-fb57-4843-a56a-9b0db57541a9 None admin] PATH_INFO: `/v3/users/1830cc1b6a8443ff8b06356e19ab38ac` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:57.924905 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ef474-fb57-4843-a56a-9b0db57541a9 None admin] RBAC: Authorizing `identity:get_user(user_id=1830cc1b6a8443ff8b06356e19ab38ac)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:57.926381 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ef474-fb57-4843-a56a-9b0db57541a9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:57.933217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 165/331] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:57 2022] GET /identity/v3/users/1830cc1b6a8443ff8b06356e19ab38ac => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:20:57.934945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ef474-fb57-4843-a56a-9b0db57541a9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:57.955208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278da782-c48b-4be6-a3e1-b55b644ec6ff None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:57.992927 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164ea19-3d53-4702-8f70-89e5b9a4aa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:57.994212 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164ea19-3d53-4702-8f70-89e5b9a4aa29 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:57.994706 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164ea19-3d53-4702-8f70-89e5b9a4aa29 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:57.995288 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164ea19-3d53-4702-8f70-89e5b9a4aa29 None admin] PATH_INFO: `/v3/users/914c71698f894d7cbf0bd6f04b96f0ff` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:57.996761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164ea19-3d53-4702-8f70-89e5b9a4aa29 None admin] RBAC: Authorizing `identity:get_user(user_id=914c71698f894d7cbf0bd6f04b96f0ff)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:57.998467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164ea19-3d53-4702-8f70-89e5b9a4aa29 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:58.000466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 167/332] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:20:57 2022] GET /identity/v3/users/914c71698f894d7cbf0bd6f04b96f0ff => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:58.006314 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164ea19-3d53-4702-8f70-89e5b9a4aa29 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:58.056139 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:58.061263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ef474-fb57-4843-a56a-9b0db57541a9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:58.094763 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:58.109980 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5127a7-a183-4012-86f6-7e27593d8a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:58.111068 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5127a7-a183-4012-86f6-7e27593d8a7d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:58.112150 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5127a7-a183-4012-86f6-7e27593d8a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:58.112638 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5127a7-a183-4012-86f6-7e27593d8a7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:58.113806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5127a7-a183-4012-86f6-7e27593d8a7d None admin] RBAC: Adding query filter params (role.id=483146fdc4ac47cf97c04a9f66c091b4, user.id=1830cc1b6a8443ff8b06356e19ab38ac, scope.system=all) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:20:58.114339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5127a7-a183-4012-86f6-7e27593d8a7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:58.116016 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5127a7-a183-4012-86f6-7e27593d8a7d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:58.131847 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 166/333] 10.222.0.40 () {56 vars in 1308 bytes} [Thu Jun 16 17:20:57 2022] GET /identity/v3/role_assignments?user.id=1830cc1b6a8443ff8b06356e19ab38ac&scope.system=all&role.id=483146fdc4ac47cf97c04a9f66c091b4 => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:58.147400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164ea19-3d53-4702-8f70-89e5b9a4aa29 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:58.198617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6ed97-4e44-4491-85e5-13c11542305f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:58.199433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6ed97-4e44-4491-85e5-13c11542305f None admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:58.199731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6ed97-4e44-4491-85e5-13c11542305f None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:58.200021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6ed97-4e44-4491-85e5-13c11542305f None admin] PATH_INFO: `/v3/system/users/914c71698f894d7cbf0bd6f04b96f0ff/roles/075fc2a9c4184b9db26b044f3d963233` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:58.207370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6ed97-4e44-4491-85e5-13c11542305f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=914c71698f894d7cbf0bd6f04b96f0ff, role_id=075fc2a9c4184b9db26b044f3d963233)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:58.208910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6ed97-4e44-4491-85e5-13c11542305f None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:58.228427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 168/334] 10.222.0.40 () {58 vars in 1334 bytes} [Thu Jun 16 17:20:58 2022] PUT /identity/v3/system/users/914c71698f894d7cbf0bd6f04b96f0ff/roles/075fc2a9c4184b9db26b044f3d963233 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:20:59.985072 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f1fda-210e-4e2e-8e1c-75cab2189d46 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:59.986460 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f1fda-210e-4e2e-8e1c-75cab2189d46 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:59.986969 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f1fda-210e-4e2e-8e1c-75cab2189d46 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:59.988570 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 167/335] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:20:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:59.994183 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85a84c-293a-473f-a017-46bf5732fbb4 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:59.995068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85a84c-293a-473f-a017-46bf5732fbb4 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:59.995568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85a84c-293a-473f-a017-46bf5732fbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:00.018238 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-9e85a84c-293a-473f-a017-46bf5732fbb4 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:00.019764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-9e85a84c-293a-473f-a017-46bf5732fbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:00.106767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 169/336] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:20:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:21:00.161529 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f1fda-210e-4e2e-8e1c-75cab2189d46 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:00.309853 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f1fda-210e-4e2e-8e1c-75cab2189d46 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:00.356357 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3701f-7aa9-4629-a398-3b9a2d0c357a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:00.357196 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3701f-7aa9-4629-a398-3b9a2d0c357a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:00.357492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3701f-7aa9-4629-a398-3b9a2d0c357a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:00.357777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3701f-7aa9-4629-a398-3b9a2d0c357a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:00.372748 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e3701f-7aa9-4629-a398-3b9a2d0c357a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:00.386528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-23e3701f-7aa9-4629-a398-3b9a2d0c357a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:21:00.386528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:00.386528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:00.386528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:00.386528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:00.386528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:00.386528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:00.386528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:00.386528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:00.386528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:00.386528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:00.386528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:00.386528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:00.386528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:00.386528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:00.386528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:00.387589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:00.387589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:00.387589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:00.387589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:00.387589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:00.387589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:00.387589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:00.387589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:00.387589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:00.387589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:00.387589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:00.387589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:00.387589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:00.387589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:00.387589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:00.387589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:00.388444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:00.388444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:00.388444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:00.388444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:00.388444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:00.388444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:00.388444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:00.388444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:00.388444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:00.388444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:00.388444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:21:00.388444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:00.388444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 168/337] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:00.399746 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85a84c-293a-473f-a017-46bf5732fbb4 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:00.477786 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:00.536921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85a84c-293a-473f-a017-46bf5732fbb4 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:00.554500 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b37f316-02c6-4b99-a8ec-4b66df5f7233 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:00.554837 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b37f316-02c6-4b99-a8ec-4b66df5f7233 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:00.555058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b37f316-02c6-4b99-a8ec-4b66df5f7233 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:00.556518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 169/338] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:00.562632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f41b6-ca60-41bf-bdd5-b4df27ca49e2 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:00.562942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f41b6-ca60-41bf-bdd5-b4df27ca49e2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:00.563206 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f41b6-ca60-41bf-bdd5-b4df27ca49e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:00.593464 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e44b7-ab1b-494a-8749-ada7fb72d756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:00.594159 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e44b7-ab1b-494a-8749-ada7fb72d756 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:00.594444 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e44b7-ab1b-494a-8749-ada7fb72d756 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:00.594724 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e44b7-ab1b-494a-8749-ada7fb72d756 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:00.595595 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e44b7-ab1b-494a-8749-ada7fb72d756 None admin] RBAC: Adding query filter params (name=member) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:00.596568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e44b7-ab1b-494a-8749-ada7fb72d756 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:00.597990 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e44b7-ab1b-494a-8749-ada7fb72d756 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:00.610227 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-571f41b6-ca60-41bf-bdd5-b4df27ca49e2 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:00.612148 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-571f41b6-ca60-41bf-bdd5-b4df27ca49e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:00.613491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 170/339] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:21:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:00.620057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e44b7-ab1b-494a-8749-ada7fb72d756 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:00.633656 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:00.742317 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 170/340] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:21:00.743677 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa9fd3-256b-4080-9d24-97f9cfddf913 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:00.743972 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa9fd3-256b-4080-9d24-97f9cfddf913 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:00.744255 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa9fd3-256b-4080-9d24-97f9cfddf913 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:00.749630 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 171/341] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:00.753038 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514985d7-74dd-4c8d-8241-a077092ad312 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:00.753537 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514985d7-74dd-4c8d-8241-a077092ad312 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:00.753987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514985d7-74dd-4c8d-8241-a077092ad312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:00.776460 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-514985d7-74dd-4c8d-8241-a077092ad312 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:00.782231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-514985d7-74dd-4c8d-8241-a077092ad312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:00.817328 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e44b7-ab1b-494a-8749-ada7fb72d756 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:00.845760 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:00.888564 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac323ac-4e72-4120-a352-0e403b222eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:00.890602 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac323ac-4e72-4120-a352-0e403b222eed None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:00.891064 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac323ac-4e72-4120-a352-0e403b222eed None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:00.891512 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac323ac-4e72-4120-a352-0e403b222eed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:00.891940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 172/342] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:00.898829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514985d7-74dd-4c8d-8241-a077092ad312 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:01.052056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac323ac-4e72-4120-a352-0e403b222eed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:01.055347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac323ac-4e72-4120-a352-0e403b222eed None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:01.079462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:01.131889 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514985d7-74dd-4c8d-8241-a077092ad312 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:01.179421 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d762be8-698c-4b78-a21b-db3cddeecc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:01.180502 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d762be8-698c-4b78-a21b-db3cddeecc19 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:01.180936 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d762be8-698c-4b78-a21b-db3cddeecc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:01.183804 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d762be8-698c-4b78-a21b-db3cddeecc19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:01.201448 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d762be8-698c-4b78-a21b-db3cddeecc19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:01.210447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-7d762be8-698c-4b78-a21b-db3cddeecc19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:21:01.210447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:01.210447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:01.210447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:01.210447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:01.210447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:01.210447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:01.210447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:01.210447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:01.210447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:01.210447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:01.210447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:01.210447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:01.210447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:01.210447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:01.210447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:01.211564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:01.211564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:01.211564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:01.211564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:01.211564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:01.211564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:01.211564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:01.211564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:01.211564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:01.211564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:01.211564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:01.211564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:01.211564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:01.211564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:01.211564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:01.211564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:01.212552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:01.212552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:01.212552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:01.212552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:01.212552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:01.212552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:01.212552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:01.212552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:01.212552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:01.212552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:01.212552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:21:01.212552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:01.214758 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 173/343] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:01.215768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d762be8-698c-4b78-a21b-db3cddeecc19 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:01.284095 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 171/344] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:21:01.285041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac323ac-4e72-4120-a352-0e403b222eed None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:01.305481 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:01.383153 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d762be8-698c-4b78-a21b-db3cddeecc19 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:01.401585 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac323ac-4e72-4120-a352-0e403b222eed None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:01.420677 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6fa50-44e4-4199-a25c-1c4c53d1c00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:01.421541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6fa50-44e4-4199-a25c-1c4c53d1c00b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:01.421832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6fa50-44e4-4199-a25c-1c4c53d1c00b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:01.422115 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6fa50-44e4-4199-a25c-1c4c53d1c00b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:01.430587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:01.434500 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6fa50-44e4-4199-a25c-1c4c53d1c00b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:01.444319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-e0b6fa50-44e4-4199-a25c-1c4c53d1c00b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:21:01.444319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:01.444319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:01.444319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:01.444319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:01.444319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:01.444319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:01.444319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:01.444319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:01.444319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:01.444319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:01.444319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:01.444319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:01.444319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:01.444319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:01.444319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:01.445526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:01.445526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:01.445526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:01.445526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:01.445526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:01.445526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:01.445526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:01.445526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:01.445526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:01.445526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:01.445526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:01.445526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:01.445526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:01.445526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:01.445526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:01.445526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:01.446640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:01.446640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:01.446640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:01.446640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:01.446640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:01.446640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:01.446640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:01.446640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:01.446640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:01.446640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:01.446640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:21:01.446640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:01.447946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d871c55-1eef-441e-b70e-5963ef3cb861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:01.448801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d871c55-1eef-441e-b70e-5963ef3cb861 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:01.449250 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d871c55-1eef-441e-b70e-5963ef3cb861 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:01.449617 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 174/345] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:21:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:01.449910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d871c55-1eef-441e-b70e-5963ef3cb861 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:01.450668 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6fa50-44e4-4199-a25c-1c4c53d1c00b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:01.451038 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d871c55-1eef-441e-b70e-5963ef3cb861 None admin] RBAC: Adding query filter params (name=member) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:01.451494 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d871c55-1eef-441e-b70e-5963ef3cb861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:01.453036 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d871c55-1eef-441e-b70e-5963ef3cb861 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:01.462959 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 172/346] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:21:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:01.464211 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d871c55-1eef-441e-b70e-5963ef3cb861 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:01.557656 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:01.584988 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6fa50-44e4-4199-a25c-1c4c53d1c00b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:01.586745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d871c55-1eef-441e-b70e-5963ef3cb861 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:01.621405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9acb5d-4ac9-4fcc-a46e-6ffd6ec31a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:01.622636 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9acb5d-4ac9-4fcc-a46e-6ffd6ec31a55 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:01.623033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9acb5d-4ac9-4fcc-a46e-6ffd6ec31a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:01.623462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9acb5d-4ac9-4fcc-a46e-6ffd6ec31a55 None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:01.624864 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9acb5d-4ac9-4fcc-a46e-6ffd6ec31a55 None admin] RBAC: Authorizing `identity:get_project(project_id=8a3d0e211dd942aaa39eed1730d17e92)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:01.626154 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94081d58-05df-4268-924e-1cbc2211b50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:01.627017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9acb5d-4ac9-4fcc-a46e-6ffd6ec31a55 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:01.627261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94081d58-05df-4268-924e-1cbc2211b50e None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:01.627731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94081d58-05df-4268-924e-1cbc2211b50e None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:01.628207 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94081d58-05df-4268-924e-1cbc2211b50e None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:01.629182 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94081d58-05df-4268-924e-1cbc2211b50e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:01.629458 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 175/347] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:21:01 2022] GET /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:01.630008 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94081d58-05df-4268-924e-1cbc2211b50e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:01.630282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9acb5d-4ac9-4fcc-a46e-6ffd6ec31a55 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:01.631816 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94081d58-05df-4268-924e-1cbc2211b50e None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:01.644323 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 173/348] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:01.645656 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94081d58-05df-4268-924e-1cbc2211b50e None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:01.705081 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:01.745954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9acb5d-4ac9-4fcc-a46e-6ffd6ec31a55 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:01.772029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94081d58-05df-4268-924e-1cbc2211b50e None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:01.781718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99305fad-a7a6-4930-a801-b169387aa800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:01.782388 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99305fad-a7a6-4930-a801-b169387aa800 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:01.782683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99305fad-a7a6-4930-a801-b169387aa800 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:01.783501 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99305fad-a7a6-4930-a801-b169387aa800 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:01.812320 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775a2b1-00c3-4a56-a6fb-e2a8d0558e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:01.813614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775a2b1-00c3-4a56-a6fb-e2a8d0558e5d None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:01.814210 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775a2b1-00c3-4a56-a6fb-e2a8d0558e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:01.814750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775a2b1-00c3-4a56-a6fb-e2a8d0558e5d None admin] PATH_INFO: `/v3/groups/f15a187fd9ce4df6b26ab36644c35421` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:01.822151 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775a2b1-00c3-4a56-a6fb-e2a8d0558e5d None admin] RBAC: Authorizing `identity:get_group(group_id=f15a187fd9ce4df6b26ab36644c35421)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:01.824000 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775a2b1-00c3-4a56-a6fb-e2a8d0558e5d None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:01.826034 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 174/349] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 17:21:01 2022] GET /identity/v3/groups/f15a187fd9ce4df6b26ab36644c35421 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:01.827173 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775a2b1-00c3-4a56-a6fb-e2a8d0558e5d None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:01.830311 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:01.880078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99305fad-a7a6-4930-a801-b169387aa800 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:01.882503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99305fad-a7a6-4930-a801-b169387aa800 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:01.952268 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775a2b1-00c3-4a56-a6fb-e2a8d0558e5d None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:01.969454 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:01.996314 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2450e3e-398a-4a53-b3e9-434ae6ac4820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:01.997481 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2450e3e-398a-4a53-b3e9-434ae6ac4820 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:01.997981 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2450e3e-398a-4a53-b3e9-434ae6ac4820 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:01.998491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2450e3e-398a-4a53-b3e9-434ae6ac4820 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:02.012211 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 176/350] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:21:02.013569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b612edac-500d-468c-8a3a-a10fdf588ee4 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:02.013866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b612edac-500d-468c-8a3a-a10fdf588ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:02.014183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b612edac-500d-468c-8a3a-a10fdf588ee4 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:02.015897 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 177/351] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:02.016558 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b612edac-500d-468c-8a3a-a10fdf588ee4 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:02.081830 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2450e3e-398a-4a53-b3e9-434ae6ac4820 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:02.084044 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2450e3e-398a-4a53-b3e9-434ae6ac4820 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:02.086438 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:02.126431 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b612edac-500d-468c-8a3a-a10fdf588ee4 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:02.170294 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd3810-3e6a-4620-8050-795f46f44694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:02.170968 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd3810-3e6a-4620-8050-795f46f44694 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:02.171252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd3810-3e6a-4620-8050-795f46f44694 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:02.171530 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd3810-3e6a-4620-8050-795f46f44694 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:02.172372 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd3810-3e6a-4620-8050-795f46f44694 None admin] RBAC: Adding query filter params (group.id=f15a187fd9ce4df6b26ab36644c35421, role.id=483146fdc4ac47cf97c04a9f66c091b4, scope.project.id=8a3d0e211dd942aaa39eed1730d17e92) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:02.172684 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd3810-3e6a-4620-8050-795f46f44694 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:02.174517 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd3810-3e6a-4620-8050-795f46f44694 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:02.182462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 178/352] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jun 16 17:21:02 2022] GET /identity/v3/role_assignments?group.id=f15a187fd9ce4df6b26ab36644c35421&scope.project.id=8a3d0e211dd942aaa39eed1730d17e92&role.id=483146fdc4ac47cf97c04a9f66c091b4 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:02.182737 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd3810-3e6a-4620-8050-795f46f44694 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:02.239183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:02.239827 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 175/353] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:21:02.242221 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9f7a9-90b2-4e69-858c-21a74586ad1f None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:02.242827 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9f7a9-90b2-4e69-858c-21a74586ad1f None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:02.243115 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9f7a9-90b2-4e69-858c-21a74586ad1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:02.266318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-cef9f7a9-90b2-4e69-858c-21a74586ad1f None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:02.268452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-cef9f7a9-90b2-4e69-858c-21a74586ad1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:02.337940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd3810-3e6a-4620-8050-795f46f44694 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:02.362378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 176/354] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:02.363751 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29040779-c3d9-4ca3-bba9-0ce748c4c0ae None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:02.364042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29040779-c3d9-4ca3-bba9-0ce748c4c0ae None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:02.364324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29040779-c3d9-4ca3-bba9-0ce748c4c0ae None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:02.365763 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 177/355] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:02.366793 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29040779-c3d9-4ca3-bba9-0ce748c4c0ae None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:02.379113 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca877e5-ab1e-4589-9df5-f7b90fda44ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:02.380437 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca877e5-ab1e-4589-9df5-f7b90fda44ab None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:02.380944 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca877e5-ab1e-4589-9df5-f7b90fda44ab None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:02.381486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca877e5-ab1e-4589-9df5-f7b90fda44ab None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:02.383087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca877e5-ab1e-4589-9df5-f7b90fda44ab None admin] RBAC: Authorizing `identity:get_project(project_id=d2415dcdc76d43c486190d67433cf01b)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:02.385193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca877e5-ab1e-4589-9df5-f7b90fda44ab None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:02.387601 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 179/356] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:21:02 2022] GET /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:02.389218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec3f3d-6a4a-4daa-9018-dca761f8d8e1 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:02.389784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec3f3d-6a4a-4daa-9018-dca761f8d8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:02.390382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec3f3d-6a4a-4daa-9018-dca761f8d8e1 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:02.392374 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 180/357] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:02.393905 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0a305-caa7-43f3-b3c8-42b325fe78a0 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:02.394424 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0a305-caa7-43f3-b3c8-42b325fe78a0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:02.394924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0a305-caa7-43f3-b3c8-42b325fe78a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:02.412954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e6c0a305-caa7-43f3-b3c8-42b325fe78a0 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:02.414462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e6c0a305-caa7-43f3-b3c8-42b325fe78a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:02.418042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:02.499866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 181/358] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:02.500815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0a305-caa7-43f3-b3c8-42b325fe78a0 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:02.513453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29040779-c3d9-4ca3-bba9-0ce748c4c0ae None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:02.537870 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:02.561026 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d40c37-879d-4996-a306-56b480560db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:02.562287 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d40c37-879d-4996-a306-56b480560db6 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:02.562287 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d40c37-879d-4996-a306-56b480560db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:02.562513 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d40c37-879d-4996-a306-56b480560db6 None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:02.563730 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d40c37-879d-4996-a306-56b480560db6 None admin] RBAC: Authorizing `identity:get_project(project_id=8a3d0e211dd942aaa39eed1730d17e92)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:02.565478 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d40c37-879d-4996-a306-56b480560db6 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:02.570407 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 178/359] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:21:02 2022] GET /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:02.570407 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d40c37-879d-4996-a306-56b480560db6 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:02.647368 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0a305-caa7-43f3-b3c8-42b325fe78a0 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:02.688031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:02.693346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d40c37-879d-4996-a306-56b480560db6 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:02.698489 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9535d041-8fdc-43f7-a714-8ced20ffd4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:02.699314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535d041-8fdc-43f7-a714-8ced20ffd4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:02.699602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535d041-8fdc-43f7-a714-8ced20ffd4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:02.699883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535d041-8fdc-43f7-a714-8ced20ffd4c4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:02.715224 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9535d041-8fdc-43f7-a714-8ced20ffd4c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:02.725017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-9535d041-8fdc-43f7-a714-8ced20ffd4c4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:21:02.725017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:02.725017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:02.725017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:02.725017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:02.725017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:02.725017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:02.725017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:02.725017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:02.725017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:02.725017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:02.725017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:02.725017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:02.725017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:02.725017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:02.725017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:02.726007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:02.726007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:02.726007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:02.726007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:02.726007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:02.726007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:02.726007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:02.726007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:02.726007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:02.726007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:02.726007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:02.726007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:02.726007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:02.726007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:02.726007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:02.726007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:02.726822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:02.726822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:02.726822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:02.726822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:02.726822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:02.726822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:02.726822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:02.726822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:02.726822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:02.726822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:02.726822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:21:02.726822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:02.726822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 182/360] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:02.727999 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc24b2-c731-465c-a362-cb60dcec23a0 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:02.727999 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc24b2-c731-465c-a362-cb60dcec23a0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:02.728551 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc24b2-c731-465c-a362-cb60dcec23a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:02.746149 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d17428-a894-4461-8604-cb199968cf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:02.747072 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d17428-a894-4461-8604-cb199968cf6c None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:02.747370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d17428-a894-4461-8604-cb199968cf6c None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:02.747659 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d17428-a894-4461-8604-cb199968cf6c None admin] PATH_INFO: `/v3/groups/f15a187fd9ce4df6b26ab36644c35421` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:02.748538 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a0fc24b2-c731-465c-a362-cb60dcec23a0 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:02.749769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a0fc24b2-c731-465c-a362-cb60dcec23a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:02.749940 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d17428-a894-4461-8604-cb199968cf6c None admin] RBAC: Authorizing `identity:get_group(group_id=f15a187fd9ce4df6b26ab36644c35421)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:02.751324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d17428-a894-4461-8604-cb199968cf6c None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:02.753128 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 179/361] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 17:21:02 2022] GET /identity/v3/groups/f15a187fd9ce4df6b26ab36644c35421 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:02.753948 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d17428-a894-4461-8604-cb199968cf6c None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:02.846122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 183/362] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:02.853190 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc24b2-c731-465c-a362-cb60dcec23a0 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:02.884109 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:02.904348 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d17428-a894-4461-8604-cb199968cf6c None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:02.952641 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df1a17-39f5-4316-9454-1ba553fbeacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:02.953523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df1a17-39f5-4316-9454-1ba553fbeacd None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:02.953817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df1a17-39f5-4316-9454-1ba553fbeacd None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:02.954102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df1a17-39f5-4316-9454-1ba553fbeacd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:02.967842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df1a17-39f5-4316-9454-1ba553fbeacd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:02.977967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-50df1a17-39f5-4316-9454-1ba553fbeacd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:21:02.977967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:02.977967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:02.977967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:02.977967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:02.977967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:02.977967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:02.977967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:02.977967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:02.977967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:02.977967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:02.977967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:02.977967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:02.977967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:02.977967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:02.977967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:02.978972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:02.978972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:02.978972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:02.978972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:02.978972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:02.978972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:02.978972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:02.978972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:02.978972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:02.978972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:02.978972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:02.978972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:02.978972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:02.978972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:02.978972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:02.978972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:02.979811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:02.979811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:02.979811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:02.979811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:02.979811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:02.979811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:02.979811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:02.979811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:02.979811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:02.979811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:02.979811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:21:02.979811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:02.979811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 180/363] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:21:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:02.980509 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df1a17-39f5-4316-9454-1ba553fbeacd None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:03.001004 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc24b2-c731-465c-a362-cb60dcec23a0 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:03.017795 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:03.044825 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8d74a-507a-4a15-800a-f03ee99c854c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:03.045859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8d74a-507a-4a15-800a-f03ee99c854c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:03.046222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8d74a-507a-4a15-800a-f03ee99c854c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:03.046583 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8d74a-507a-4a15-800a-f03ee99c854c None admin] PATH_INFO: `/v3/groups/f15a187fd9ce4df6b26ab36644c35421` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:03.047870 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8d74a-507a-4a15-800a-f03ee99c854c None admin] RBAC: Authorizing `identity:get_group(group_id=f15a187fd9ce4df6b26ab36644c35421)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:03.051022 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8d74a-507a-4a15-800a-f03ee99c854c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:03.056292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 184/364] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 17:21:02 2022] GET /identity/v3/groups/f15a187fd9ce4df6b26ab36644c35421 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:03.056292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8d74a-507a-4a15-800a-f03ee99c854c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:03.122160 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df1a17-39f5-4316-9454-1ba553fbeacd None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:03.167243 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:03.177764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a738ef-87c3-4073-9a2e-d2d8afbd9bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:03.178456 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a738ef-87c3-4073-9a2e-d2d8afbd9bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:03.178962 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a738ef-87c3-4073-9a2e-d2d8afbd9bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:03.179202 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a738ef-87c3-4073-9a2e-d2d8afbd9bb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:03.179979 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a738ef-87c3-4073-9a2e-d2d8afbd9bb4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:03.180300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a738ef-87c3-4073-9a2e-d2d8afbd9bb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:03.181877 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a738ef-87c3-4073-9a2e-d2d8afbd9bb4 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:03.190521 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8d74a-507a-4a15-800a-f03ee99c854c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:03.193998 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 181/365] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:21:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:03.197171 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a738ef-87c3-4073-9a2e-d2d8afbd9bb4 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:03.237950 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f0e2f-6ed1-4218-ba26-7b827fdf00d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:03.238633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f0e2f-6ed1-4218-ba26-7b827fdf00d0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:03.238919 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f0e2f-6ed1-4218-ba26-7b827fdf00d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:03.239198 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f0e2f-6ed1-4218-ba26-7b827fdf00d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:03.240044 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f0e2f-6ed1-4218-ba26-7b827fdf00d0 None admin] RBAC: Adding query filter params (group.id=f15a187fd9ce4df6b26ab36644c35421, role.id=483146fdc4ac47cf97c04a9f66c091b4, scope.project.id=d2415dcdc76d43c486190d67433cf01b) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:03.240469 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f0e2f-6ed1-4218-ba26-7b827fdf00d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:03.241907 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f0e2f-6ed1-4218-ba26-7b827fdf00d0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:03.247897 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 185/366] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jun 16 17:21:03 2022] GET /identity/v3/role_assignments?group.id=f15a187fd9ce4df6b26ab36644c35421&scope.project.id=d2415dcdc76d43c486190d67433cf01b&role.id=483146fdc4ac47cf97c04a9f66c091b4 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:03.248694 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f0e2f-6ed1-4218-ba26-7b827fdf00d0 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:03.322826 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:03.339581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a738ef-87c3-4073-9a2e-d2d8afbd9bb4 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:03.400098 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2690277-7780-4371-abc4-eec6d2e0cd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:03.400833 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2690277-7780-4371-abc4-eec6d2e0cd64 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:03.401279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2690277-7780-4371-abc4-eec6d2e0cd64 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:03.401664 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2690277-7780-4371-abc4-eec6d2e0cd64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:03.416139 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2690277-7780-4371-abc4-eec6d2e0cd64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:03.426057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-e2690277-7780-4371-abc4-eec6d2e0cd64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:21:03.426057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:03.426057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:03.426057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:03.426057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:03.426057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:03.426057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:03.426057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:03.426057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:03.426057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:03.426057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:03.426057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:03.426057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:03.426057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:03.426057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:03.426057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:03.427220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:03.427220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:03.427220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:03.427220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:03.427220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:03.427220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:03.427220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:03.427220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:03.427220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:03.427220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:03.427220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:03.427220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:03.427220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:03.427220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:03.427220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:03.427220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:03.428253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:03.428253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:03.428253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:03.428253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:03.428253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:03.428253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:03.428253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:03.428253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:03.428253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:03.428253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:03.428253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:21:03.428253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:03.430504 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 182/367] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:21:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:03.431555 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2690277-7780-4371-abc4-eec6d2e0cd64 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:03.433490 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f0e2f-6ed1-4218-ba26-7b827fdf00d0 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:03.490121 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15a411-4012-40cc-b7d6-ba78254651e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:03.491299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15a411-4012-40cc-b7d6-ba78254651e6 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:03.491299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15a411-4012-40cc-b7d6-ba78254651e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:03.491299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15a411-4012-40cc-b7d6-ba78254651e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:03.492013 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15a411-4012-40cc-b7d6-ba78254651e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:03.492330 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15a411-4012-40cc-b7d6-ba78254651e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:03.494069 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15a411-4012-40cc-b7d6-ba78254651e6 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:03.503531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 186/368] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:03.504320 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15a411-4012-40cc-b7d6-ba78254651e6 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:03.533908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:03.563741 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2690277-7780-4371-abc4-eec6d2e0cd64 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:03.629632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc919bee-6b49-4445-881e-d8050fd83661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:03.630267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc919bee-6b49-4445-881e-d8050fd83661 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:03.630559 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc919bee-6b49-4445-881e-d8050fd83661 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:03.630841 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc919bee-6b49-4445-881e-d8050fd83661 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:03.631686 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc919bee-6b49-4445-881e-d8050fd83661 None admin] RBAC: Adding query filter params (group.id=f15a187fd9ce4df6b26ab36644c35421, role.id=1423a24997e840608b6c522958481fde, scope.project.id=8a3d0e211dd942aaa39eed1730d17e92) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:03.632007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc919bee-6b49-4445-881e-d8050fd83661 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:03.632250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15a411-4012-40cc-b7d6-ba78254651e6 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:03.634953 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc919bee-6b49-4445-881e-d8050fd83661 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:03.641499 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 183/369] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jun 16 17:21:03 2022] GET /identity/v3/role_assignments?group.id=f15a187fd9ce4df6b26ab36644c35421&scope.project.id=8a3d0e211dd942aaa39eed1730d17e92&role.id=1423a24997e840608b6c522958481fde => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:03.642860 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc919bee-6b49-4445-881e-d8050fd83661 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:03.675682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:03.682516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca9c06-59f6-4d68-9576-36ce9b7bda02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:03.683445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca9c06-59f6-4d68-9576-36ce9b7bda02 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:03.683736 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca9c06-59f6-4d68-9576-36ce9b7bda02 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:03.684020 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca9c06-59f6-4d68-9576-36ce9b7bda02 None admin] PATH_INFO: `/v3/users/914c71698f894d7cbf0bd6f04b96f0ff` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:03.693478 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca9c06-59f6-4d68-9576-36ce9b7bda02 None admin] RBAC: Authorizing `identity:get_user(user_id=914c71698f894d7cbf0bd6f04b96f0ff)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:03.695379 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca9c06-59f6-4d68-9576-36ce9b7bda02 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:03.697799 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 187/370] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:21:03 2022] GET /identity/v3/users/914c71698f894d7cbf0bd6f04b96f0ff => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:03.698911 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca9c06-59f6-4d68-9576-36ce9b7bda02 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:03.840599 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc919bee-6b49-4445-881e-d8050fd83661 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:03.865553 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:03.903110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca9c06-59f6-4d68-9576-36ce9b7bda02 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:03.914468 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64945186-7fad-49c9-9c3a-54e0a410c408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:03.915166 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64945186-7fad-49c9-9c3a-54e0a410c408 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:03.915459 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64945186-7fad-49c9-9c3a-54e0a410c408 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:03.915745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64945186-7fad-49c9-9c3a-54e0a410c408 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:03.916512 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64945186-7fad-49c9-9c3a-54e0a410c408 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:03.916825 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64945186-7fad-49c9-9c3a-54e0a410c408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:03.918264 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64945186-7fad-49c9-9c3a-54e0a410c408 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:03.930942 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 184/371] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:03.931803 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64945186-7fad-49c9-9c3a-54e0a410c408 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:03.973049 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e20fa-558c-40f3-953d-921810fc9826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:03.973765 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e20fa-558c-40f3-953d-921810fc9826 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:03.974062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e20fa-558c-40f3-953d-921810fc9826 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:03.974348 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e20fa-558c-40f3-953d-921810fc9826 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:04.088240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e20fa-558c-40f3-953d-921810fc9826 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:04.089830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e20fa-558c-40f3-953d-921810fc9826 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:04.112033 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:04.181612 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:04.199392 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64945186-7fad-49c9-9c3a-54e0a410c408 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:04.206351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 188/372] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:21:04.207442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e20fa-558c-40f3-953d-921810fc9826 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:04.303110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ea920-05ea-4254-a5bb-8544940d0821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:04.304203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ea920-05ea-4254-a5bb-8544940d0821 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:04.304698 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ea920-05ea-4254-a5bb-8544940d0821 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:04.305218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ea920-05ea-4254-a5bb-8544940d0821 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:04.306259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ea920-05ea-4254-a5bb-8544940d0821 None admin] RBAC: Adding query filter params (role.id=075fc2a9c4184b9db26b044f3d963233, user.id=914c71698f894d7cbf0bd6f04b96f0ff, scope.system=all) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:04.306770 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ea920-05ea-4254-a5bb-8544940d0821 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:04.308525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ea920-05ea-4254-a5bb-8544940d0821 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:04.331464 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 185/373] 10.222.0.40 () {56 vars in 1308 bytes} [Thu Jun 16 17:21:03 2022] GET /identity/v3/role_assignments?user.id=914c71698f894d7cbf0bd6f04b96f0ff&scope.system=all&role.id=075fc2a9c4184b9db26b044f3d963233 => generated 510 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:04.332818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ea920-05ea-4254-a5bb-8544940d0821 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:04.335885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e20fa-558c-40f3-953d-921810fc9826 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:04.380423 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32884d28-576b-4e5c-a692-be8e2367a8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:04.381091 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32884d28-576b-4e5c-a692-be8e2367a8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:04.381491 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32884d28-576b-4e5c-a692-be8e2367a8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:04.381775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32884d28-576b-4e5c-a692-be8e2367a8d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:04.454490 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:04.480646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32884d28-576b-4e5c-a692-be8e2367a8d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:04.482440 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32884d28-576b-4e5c-a692-be8e2367a8d7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:04.549223 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ea920-05ea-4254-a5bb-8544940d0821 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:04.610581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 189/374] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:21:04.618453 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32884d28-576b-4e5c-a692-be8e2367a8d7 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:04.675120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ca352-da2c-4032-91ec-c86dc63d59d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:04.676014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ca352-da2c-4032-91ec-c86dc63d59d5 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:04.676304 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ca352-da2c-4032-91ec-c86dc63d59d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:04.676587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ca352-da2c-4032-91ec-c86dc63d59d5 None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:04.679058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ca352-da2c-4032-91ec-c86dc63d59d5 None admin] RBAC: Authorizing `identity:get_project(project_id=d2415dcdc76d43c486190d67433cf01b)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:04.680533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ca352-da2c-4032-91ec-c86dc63d59d5 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:04.691177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 186/375] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:21:04 2022] GET /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b => generated 302 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:04.702233 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ca352-da2c-4032-91ec-c86dc63d59d5 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:04.705855 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:04.790586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:04.832198 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32884d28-576b-4e5c-a692-be8e2367a8d7 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:04.877905 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8062541-65c2-4017-b011-20d484735fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:04.878842 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8062541-65c2-4017-b011-20d484735fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:04.879022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ca352-da2c-4032-91ec-c86dc63d59d5 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:04.879148 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8062541-65c2-4017-b011-20d484735fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:04.879408 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8062541-65c2-4017-b011-20d484735fd4 None admin] PATH_INFO: `/v3/projects/26ada52fc9b44bcbbe7f2cc5f37ab303` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:04.882037 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8062541-65c2-4017-b011-20d484735fd4 None admin] RBAC: Authorizing `identity:get_project(project_id=26ada52fc9b44bcbbe7f2cc5f37ab303)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:04.883516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8062541-65c2-4017-b011-20d484735fd4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:04.885861 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 190/376] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:21:04 2022] GET /identity/v3/projects/26ada52fc9b44bcbbe7f2cc5f37ab303 => generated 344 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:04.887107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf9aaa-0c35-4d09-a781-868d27b22ed9 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:04.887401 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf9aaa-0c35-4d09-a781-868d27b22ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:04.887685 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf9aaa-0c35-4d09-a781-868d27b22ed9 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:04.889197 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 191/377] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:04.894424 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf9aaa-0c35-4d09-a781-868d27b22ed9 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:04.930864 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6711e9-946a-4e0e-9d50-bdfe144b69a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:04.931747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6711e9-946a-4e0e-9d50-bdfe144b69a4 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:04.932040 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6711e9-946a-4e0e-9d50-bdfe144b69a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:04.932325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6711e9-946a-4e0e-9d50-bdfe144b69a4 None admin] PATH_INFO: `/v3/groups/f15a187fd9ce4df6b26ab36644c35421` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:04.934854 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6711e9-946a-4e0e-9d50-bdfe144b69a4 None admin] RBAC: Authorizing `identity:get_group(group_id=f15a187fd9ce4df6b26ab36644c35421)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:04.936316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6711e9-946a-4e0e-9d50-bdfe144b69a4 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:04.938048 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 187/378] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 17:21:04 2022] GET /identity/v3/groups/f15a187fd9ce4df6b26ab36644c35421 => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:04.939275 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f6cac-e4cc-4d57-b310-e8ab84d90b36 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:04.939568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f6cac-e4cc-4d57-b310-e8ab84d90b36 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:04.939865 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f6cac-e4cc-4d57-b310-e8ab84d90b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:04.958691 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-148f6cac-e4cc-4d57-b310-e8ab84d90b36 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:04.959870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-148f6cac-e4cc-4d57-b310-e8ab84d90b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:05.022006 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf9aaa-0c35-4d09-a781-868d27b22ed9 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:05.037512 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:05.041207 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 188/379] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:05.046690 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f6cac-e4cc-4d57-b310-e8ab84d90b36 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:05.085027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34def195-7e10-4e08-aa82-1d737115b8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:05.085957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34def195-7e10-4e08-aa82-1d737115b8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:05.086248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34def195-7e10-4e08-aa82-1d737115b8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:05.086532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34def195-7e10-4e08-aa82-1d737115b8e6 None admin] PATH_INFO: `/v3/groups/33013e492f804e768fade85df43ac33d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:05.093451 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34def195-7e10-4e08-aa82-1d737115b8e6 None admin] RBAC: Authorizing `identity:get_group(group_id=33013e492f804e768fade85df43ac33d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:05.094923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34def195-7e10-4e08-aa82-1d737115b8e6 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:05.098266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 192/380] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 17:21:04 2022] GET /identity/v3/groups/33013e492f804e768fade85df43ac33d => generated 229 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:05.099069 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34def195-7e10-4e08-aa82-1d737115b8e6 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:05.115558 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:05.160683 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f6cac-e4cc-4d57-b310-e8ab84d90b36 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:05.199609 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7d210-014a-4092-97d1-48bf2d1fd746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:05.200398 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7d210-014a-4092-97d1-48bf2d1fd746 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:05.200562 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7d210-014a-4092-97d1-48bf2d1fd746 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:05.200849 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7d210-014a-4092-97d1-48bf2d1fd746 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:05.201820 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7d210-014a-4092-97d1-48bf2d1fd746 None admin] RBAC: Adding query filter params (group.id=f15a187fd9ce4df6b26ab36644c35421, role.id=1423a24997e840608b6c522958481fde, scope.project.id=d2415dcdc76d43c486190d67433cf01b) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:05.202142 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7d210-014a-4092-97d1-48bf2d1fd746 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:05.203618 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7d210-014a-4092-97d1-48bf2d1fd746 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:05.210731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 189/381] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jun 16 17:21:05 2022] GET /identity/v3/role_assignments?group.id=f15a187fd9ce4df6b26ab36644c35421&scope.project.id=d2415dcdc76d43c486190d67433cf01b&role.id=1423a24997e840608b6c522958481fde => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:05.211490 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7d210-014a-4092-97d1-48bf2d1fd746 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:05.223061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34def195-7e10-4e08-aa82-1d737115b8e6 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:05.283354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:05.294192 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c901cf3-4bc2-4fad-9a5c-d2a44d870d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:05.295023 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c901cf3-4bc2-4fad-9a5c-d2a44d870d13 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:05.295467 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c901cf3-4bc2-4fad-9a5c-d2a44d870d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:05.295598 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c901cf3-4bc2-4fad-9a5c-d2a44d870d13 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:05.318440 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c901cf3-4bc2-4fad-9a5c-d2a44d870d13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:05.327591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7d210-014a-4092-97d1-48bf2d1fd746 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:05.334716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-7c901cf3-4bc2-4fad-9a5c-d2a44d870d13 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:21:05.334716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:05.334716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:05.334716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:05.334716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:05.334716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:05.334716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:05.334716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:05.334716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:05.334716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:05.334716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:05.334716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:05.334716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:05.334716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:05.334716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:05.334716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:05.335706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:05.335706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:05.335706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:05.335706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:05.335706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:05.335706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:05.335706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:05.335706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:05.335706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:05.335706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:05.335706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:05.335706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:05.335706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:05.335706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:05.335706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:05.335706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:05.336516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:05.336516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:05.336516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:05.336516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:05.336516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:05.336516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:05.336516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:05.336516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:05.336516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:05.336516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:05.336516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:21:05.336516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:05.336516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 193/382] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:05.342187 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c901cf3-4bc2-4fad-9a5c-d2a44d870d13 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:05.364495 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af2ce9-c90b-4d98-856e-f415ba869ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:05.365212 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af2ce9-c90b-4d98-856e-f415ba869ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:05.365512 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af2ce9-c90b-4d98-856e-f415ba869ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:05.365858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af2ce9-c90b-4d98-856e-f415ba869ef5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:05.366769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af2ce9-c90b-4d98-856e-f415ba869ef5 None admin] RBAC: Adding query filter params (group.id=33013e492f804e768fade85df43ac33d, role.id=25a16d8ea4084e9eacdebb7a4dcd2337, scope.project.id=26ada52fc9b44bcbbe7f2cc5f37ab303) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:05.367100 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af2ce9-c90b-4d98-856e-f415ba869ef5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:05.368629 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af2ce9-c90b-4d98-856e-f415ba869ef5 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:05.381712 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 190/383] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jun 16 17:21:05 2022] GET /identity/v3/role_assignments?group.id=33013e492f804e768fade85df43ac33d&scope.project.id=26ada52fc9b44bcbbe7f2cc5f37ab303&role.id=25a16d8ea4084e9eacdebb7a4dcd2337 => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:05.462618 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:05.505985 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c901cf3-4bc2-4fad-9a5c-d2a44d870d13 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:05.549672 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6732d877-4a22-4b57-9a53-5740d9e37837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:05.550394 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732d877-4a22-4b57-9a53-5740d9e37837 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:05.550688 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732d877-4a22-4b57-9a53-5740d9e37837 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:05.550973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732d877-4a22-4b57-9a53-5740d9e37837 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:05.551767 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6732d877-4a22-4b57-9a53-5740d9e37837 None admin] RBAC: Adding query filter params (name=member) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:05.552088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6732d877-4a22-4b57-9a53-5740d9e37837 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:05.553550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6732d877-4a22-4b57-9a53-5740d9e37837 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:05.563644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 194/384] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:21:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:05.570717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af2ce9-c90b-4d98-856e-f415ba869ef5 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:05.715934 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af2ce9-c90b-4d98-856e-f415ba869ef5 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:05.746292 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:05.755571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee99b22-e82f-43ba-966c-cbd62d8d74f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:05.756457 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee99b22-e82f-43ba-966c-cbd62d8d74f7 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:05.756999 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee99b22-e82f-43ba-966c-cbd62d8d74f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:05.757731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee99b22-e82f-43ba-966c-cbd62d8d74f7 None admin] PATH_INFO: `/v3/groups/f15a187fd9ce4df6b26ab36644c35421` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:05.758947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee99b22-e82f-43ba-966c-cbd62d8d74f7 None admin] RBAC: Authorizing `identity:get_group(group_id=f15a187fd9ce4df6b26ab36644c35421)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:05.760677 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee99b22-e82f-43ba-966c-cbd62d8d74f7 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:05.762684 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 191/385] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 17:21:05 2022] GET /identity/v3/groups/f15a187fd9ce4df6b26ab36644c35421 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:05.769434 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6732d877-4a22-4b57-9a53-5740d9e37837 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:05.924221 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864abe3-ff39-4521-a536-022b2b79e355 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:05.924994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864abe3-ff39-4521-a536-022b2b79e355 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:05.924994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864abe3-ff39-4521-a536-022b2b79e355 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:05.926910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 192/386] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:05.932040 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19766673-f7c3-4057-b43b-66de1ba8e702 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:05.932375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19766673-f7c3-4057-b43b-66de1ba8e702 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:05.932780 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19766673-f7c3-4057-b43b-66de1ba8e702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:05.936447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6732d877-4a22-4b57-9a53-5740d9e37837 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:05.969651 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-19766673-f7c3-4057-b43b-66de1ba8e702 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:05.971606 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-19766673-f7c3-4057-b43b-66de1ba8e702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:05.979143 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb9ae9-4146-4e6b-8ef9-98792527fee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:05.980258 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb9ae9-4146-4e6b-8ef9-98792527fee1 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:05.980683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb9ae9-4146-4e6b-8ef9-98792527fee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:05.981094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb9ae9-4146-4e6b-8ef9-98792527fee1 None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:05.982544 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb9ae9-4146-4e6b-8ef9-98792527fee1 None admin] RBAC: Authorizing `identity:get_project(project_id=8a3d0e211dd942aaa39eed1730d17e92)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:05.984135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb9ae9-4146-4e6b-8ef9-98792527fee1 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:05.987554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 195/387] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:21:05 2022] GET /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:05.994768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb9ae9-4146-4e6b-8ef9-98792527fee1 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:05.997963 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:06.090109 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 193/388] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:06.118766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb9ae9-4146-4e6b-8ef9-98792527fee1 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:06.129528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19766673-f7c3-4057-b43b-66de1ba8e702 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:06.160449 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c78db-a975-45e7-a6c8-6d55c0031705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:06.161674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c78db-a975-45e7-a6c8-6d55c0031705 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:06.162098 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c78db-a975-45e7-a6c8-6d55c0031705 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:06.162511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c78db-a975-45e7-a6c8-6d55c0031705 None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92/groups/f15a187fd9ce4df6b26ab36644c35421/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:06.164728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483c78db-a975-45e7-a6c8-6d55c0031705 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a3d0e211dd942aaa39eed1730d17e92, group_id=f15a187fd9ce4df6b26ab36644c35421, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:06.166382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483c78db-a975-45e7-a6c8-6d55c0031705 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:06.183144 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 196/389] 10.222.0.40 () {58 vars in 1442 bytes} [Thu Jun 16 17:21:05 2022] PUT /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92/groups/f15a187fd9ce4df6b26ab36644c35421/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:21:06.291644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19766673-f7c3-4057-b43b-66de1ba8e702 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:06.326538 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:06.335234 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054412e-fa79-4a04-ad5f-963770ac4ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:06.336083 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054412e-fa79-4a04-ad5f-963770ac4ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:06.336374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054412e-fa79-4a04-ad5f-963770ac4ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:06.336656 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054412e-fa79-4a04-ad5f-963770ac4ec7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:06.350355 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054412e-fa79-4a04-ad5f-963770ac4ec7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:06.358413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-7054412e-fa79-4a04-ad5f-963770ac4ec7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:21:06.358413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:06.358413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:06.358413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:06.358413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:06.358413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:06.358413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:06.358413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:06.358413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:06.358413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:06.358413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:06.358413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:06.358413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:06.358413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:06.358413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:06.358413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:06.359429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:06.359429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:06.359429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:06.359429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:06.359429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:06.359429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:06.359429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:06.359429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:06.359429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:06.359429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:06.359429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:06.359429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:06.359429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:06.359429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:06.359429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:06.359429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:06.360294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:06.360294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:06.360294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:06.360294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:06.360294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:06.360294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:06.360294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:06.360294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:06.360294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:06.360294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:06.360294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:21:06.360294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:06.360294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 194/390] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:06.366168 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c78db-a975-45e7-a6c8-6d55c0031705 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:06.398650 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d4e62-ff0e-4192-82d7-e929bd9d5932 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:06.398996 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d4e62-ff0e-4192-82d7-e929bd9d5932 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:06.399218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d4e62-ff0e-4192-82d7-e929bd9d5932 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:06.401090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 195/391] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:21:06.405911 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3919a-fe9b-4ad4-96f2-8d9b6e9f759d None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:06.406164 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3919a-fe9b-4ad4-96f2-8d9b6e9f759d None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:06.406440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3919a-fe9b-4ad4-96f2-8d9b6e9f759d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:06.436941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-03e3919a-fe9b-4ad4-96f2-8d9b6e9f759d None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:06.438202 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-03e3919a-fe9b-4ad4-96f2-8d9b6e9f759d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:06.496900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:06.549186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c78db-a975-45e7-a6c8-6d55c0031705 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:06.554084 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 196/392] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:21:06.587077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf397256-8570-4b78-9da9-2041b45e8025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:06.587770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf397256-8570-4b78-9da9-2041b45e8025 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:06.588066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf397256-8570-4b78-9da9-2041b45e8025 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:06.588352 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf397256-8570-4b78-9da9-2041b45e8025 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:06.588675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3919a-fe9b-4ad4-96f2-8d9b6e9f759d None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:06.589122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf397256-8570-4b78-9da9-2041b45e8025 None admin] RBAC: Adding query filter params (name=member) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:06.589473 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf397256-8570-4b78-9da9-2041b45e8025 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:06.590891 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf397256-8570-4b78-9da9-2041b45e8025 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:06.600755 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 197/393] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:21:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:06.606993 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf397256-8570-4b78-9da9-2041b45e8025 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:06.713079 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf397256-8570-4b78-9da9-2041b45e8025 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:06.715818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3919a-fe9b-4ad4-96f2-8d9b6e9f759d None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:06.736845 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:06.737771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:06.748673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d7f97-7f92-4c1b-b951-0b08aa962f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:06.749504 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d7f97-7f92-4c1b-b951-0b08aa962f67 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:06.749792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d7f97-7f92-4c1b-b951-0b08aa962f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:06.750069 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d7f97-7f92-4c1b-b951-0b08aa962f67 None admin] PATH_INFO: `/v3/groups/f15a187fd9ce4df6b26ab36644c35421` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:06.751335 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d7f97-7f92-4c1b-b951-0b08aa962f67 None admin] RBAC: Authorizing `identity:get_group(group_id=f15a187fd9ce4df6b26ab36644c35421)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:06.752806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d7f97-7f92-4c1b-b951-0b08aa962f67 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:06.754786 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 198/394] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 17:21:06 2022] GET /identity/v3/groups/f15a187fd9ce4df6b26ab36644c35421 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:06.760396 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d7f97-7f92-4c1b-b951-0b08aa962f67 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:06.761687 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd9e0b-f416-4938-9cb4-6096b844d125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:06.762746 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd9e0b-f416-4938-9cb4-6096b844d125 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:06.763246 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd9e0b-f416-4938-9cb4-6096b844d125 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:06.763760 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd9e0b-f416-4938-9cb4-6096b844d125 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:06.776673 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd9e0b-f416-4938-9cb4-6096b844d125 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:06.786955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-44bd9e0b-f416-4938-9cb4-6096b844d125 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:21:06.786955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:06.786955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:06.786955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:06.786955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:06.786955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:06.786955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:06.786955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:06.786955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:06.786955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:06.786955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:06.786955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:06.786955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:06.786955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:06.786955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:06.786955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:06.787875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:06.787875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:06.787875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:06.787875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:06.787875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:06.787875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:06.787875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:06.787875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:06.787875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:06.787875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:06.787875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:06.787875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:06.787875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:06.787875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:06.787875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:06.787875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:06.788689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:06.788689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:06.788689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:06.788689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:06.788689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:06.788689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:06.788689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:06.788689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:06.788689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:06.788689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:06.788689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:21:06.788689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:06.792056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 197/395] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:21:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:06.798216 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd9e0b-f416-4938-9cb4-6096b844d125 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:06.891355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d7f97-7f92-4c1b-b951-0b08aa962f67 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:06.918849 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd9e0b-f416-4938-9cb4-6096b844d125 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:06.957347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765a9e1-d4dd-424a-b3cc-293491ad108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:06.958403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765a9e1-d4dd-424a-b3cc-293491ad108c None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:06.958928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765a9e1-d4dd-424a-b3cc-293491ad108c None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:06.959427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765a9e1-d4dd-424a-b3cc-293491ad108c None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:06.960453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1765a9e1-d4dd-424a-b3cc-293491ad108c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:06.961008 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1765a9e1-d4dd-424a-b3cc-293491ad108c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:06.962696 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1765a9e1-d4dd-424a-b3cc-293491ad108c None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:06.974182 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 198/396] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:06.980071 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765a9e1-d4dd-424a-b3cc-293491ad108c None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:07.010291 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9459d-169e-4dde-ad8a-a25314616f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:07.011118 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9459d-169e-4dde-ad8a-a25314616f96 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:07.011409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9459d-169e-4dde-ad8a-a25314616f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:07.011717 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9459d-169e-4dde-ad8a-a25314616f96 None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:07.018210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9459d-169e-4dde-ad8a-a25314616f96 None admin] RBAC: Authorizing `identity:get_project(project_id=d2415dcdc76d43c486190d67433cf01b)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:07.019741 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9459d-169e-4dde-ad8a-a25314616f96 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:07.021668 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 199/397] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:21:06 2022] GET /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:07.027834 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9459d-169e-4dde-ad8a-a25314616f96 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:07.034154 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:07.102736 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765a9e1-d4dd-424a-b3cc-293491ad108c None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:07.142750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c8ed2-7a47-4f21-95c8-96f8a4420077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:07.144513 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c8ed2-7a47-4f21-95c8-96f8a4420077 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:07.145030 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c8ed2-7a47-4f21-95c8-96f8a4420077 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:07.145569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c8ed2-7a47-4f21-95c8-96f8a4420077 None admin] PATH_INFO: `/v3/groups/f15a187fd9ce4df6b26ab36644c35421` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:07.147124 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c8ed2-7a47-4f21-95c8-96f8a4420077 None admin] RBAC: Authorizing `identity:get_group(group_id=f15a187fd9ce4df6b26ab36644c35421)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:07.148885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c8ed2-7a47-4f21-95c8-96f8a4420077 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:07.151058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 199/398] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 17:21:06 2022] GET /identity/v3/groups/f15a187fd9ce4df6b26ab36644c35421 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:07.156782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c8ed2-7a47-4f21-95c8-96f8a4420077 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:07.169885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:07.260015 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9459d-169e-4dde-ad8a-a25314616f96 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:07.324482 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c8ed2-7a47-4f21-95c8-96f8a4420077 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:07.328038 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45123d5f-cd6b-45dd-a06a-7db01785ab14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:07.328929 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45123d5f-cd6b-45dd-a06a-7db01785ab14 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:07.329243 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45123d5f-cd6b-45dd-a06a-7db01785ab14 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:07.330064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45123d5f-cd6b-45dd-a06a-7db01785ab14 None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b/groups/f15a187fd9ce4df6b26ab36644c35421/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:07.338807 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45123d5f-cd6b-45dd-a06a-7db01785ab14 None admin] RBAC: Authorizing `identity:create_grant(project_id=d2415dcdc76d43c486190d67433cf01b, group_id=f15a187fd9ce4df6b26ab36644c35421, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:07.339364 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:07.340358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45123d5f-cd6b-45dd-a06a-7db01785ab14 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:07.376983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d025b-8b25-43a4-bdbe-e22dd3fbf097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:07.377913 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d025b-8b25-43a4-bdbe-e22dd3fbf097 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:07.378249 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d025b-8b25-43a4-bdbe-e22dd3fbf097 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:07.378495 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 200/399] 10.222.0.40 () {58 vars in 1442 bytes} [Thu Jun 16 17:21:07 2022] PUT /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b/groups/f15a187fd9ce4df6b26ab36644c35421/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:21:07.379007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d025b-8b25-43a4-bdbe-e22dd3fbf097 None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:07.386099 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d025b-8b25-43a4-bdbe-e22dd3fbf097 None admin] RBAC: Authorizing `identity:get_project(project_id=8a3d0e211dd942aaa39eed1730d17e92)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:07.387594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d025b-8b25-43a4-bdbe-e22dd3fbf097 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:07.389759 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 200/400] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:21:07 2022] GET /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:07.398041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45123d5f-cd6b-45dd-a06a-7db01785ab14 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:07.574413 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45123d5f-cd6b-45dd-a06a-7db01785ab14 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:07.597252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:07.613534 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0d2bc-d3ab-4482-8e26-2519ed0ac6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:07.615109 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0d2bc-d3ab-4482-8e26-2519ed0ac6c8 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:07.615703 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0d2bc-d3ab-4482-8e26-2519ed0ac6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:07.615992 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0d2bc-d3ab-4482-8e26-2519ed0ac6c8 None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92/groups/f15a187fd9ce4df6b26ab36644c35421/roles/1423a24997e840608b6c522958481fde` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:07.618480 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa0d2bc-d3ab-4482-8e26-2519ed0ac6c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a3d0e211dd942aaa39eed1730d17e92, group_id=f15a187fd9ce4df6b26ab36644c35421, role_id=1423a24997e840608b6c522958481fde)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:07.619940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa0d2bc-d3ab-4482-8e26-2519ed0ac6c8 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:07.636174 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 201/401] 10.222.0.40 () {58 vars in 1442 bytes} [Thu Jun 16 17:21:07 2022] PUT /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92/groups/f15a187fd9ce4df6b26ab36644c35421/roles/1423a24997e840608b6c522958481fde => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:21:08.174703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3fd991-d661-4149-9f8b-7d18eac432ce None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:08.175126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3fd991-d661-4149-9f8b-7d18eac432ce None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:08.175320 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3fd991-d661-4149-9f8b-7d18eac432ce None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:08.176948 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 201/402] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:08.182121 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae6e1a-6c1e-4a25-afe8-e866c23d4113 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:08.182419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae6e1a-6c1e-4a25-afe8-e866c23d4113 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:08.182702 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae6e1a-6c1e-4a25-afe8-e866c23d4113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:08.206194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-aaae6e1a-6c1e-4a25-afe8-e866c23d4113 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:08.207420 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-aaae6e1a-6c1e-4a25-afe8-e866c23d4113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:08.299431 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 202/403] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:08.343498 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3fd991-d661-4149-9f8b-7d18eac432ce None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:08.389907 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84b9b2-e6b1-4c15-800a-e9d8ef8ddae3 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:08.390448 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84b9b2-e6b1-4c15-800a-e9d8ef8ddae3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:08.390448 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84b9b2-e6b1-4c15-800a-e9d8ef8ddae3 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:08.392152 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 203/404] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:08.397224 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527ffb3-1ac2-472f-a582-11db80785ae0 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:08.397520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527ffb3-1ac2-472f-a582-11db80785ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:08.397807 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527ffb3-1ac2-472f-a582-11db80785ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:08.423494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-c527ffb3-1ac2-472f-a582-11db80785ae0 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:08.424773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c527ffb3-1ac2-472f-a582-11db80785ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:08.492638 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3fd991-d661-4149-9f8b-7d18eac432ce None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:08.558826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 204/405] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:21:08.562196 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96724476-80e7-4ac8-8da2-83e95b5d438c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:08.563033 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96724476-80e7-4ac8-8da2-83e95b5d438c None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:08.563325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96724476-80e7-4ac8-8da2-83e95b5d438c None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:08.563612 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96724476-80e7-4ac8-8da2-83e95b5d438c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:08.581414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527ffb3-1ac2-472f-a582-11db80785ae0 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:08.590451 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96724476-80e7-4ac8-8da2-83e95b5d438c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:08.604427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-96724476-80e7-4ac8-8da2-83e95b5d438c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:21:08.604427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:08.604427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:08.604427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:08.604427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:08.604427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:08.604427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:08.604427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:08.604427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:08.604427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:08.604427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:08.604427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:08.604427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:08.604427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:08.604427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:08.604427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:08.605371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:08.605371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:08.605371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:08.605371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:08.605371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:08.605371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:08.605371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:08.605371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:08.605371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:08.605371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:08.605371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:08.605371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:08.605371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:08.605371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:08.605371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:08.605371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:08.606198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:08.606198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:08.606198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:08.606198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:08.606198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:08.606198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:08.606198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:08.606198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:08.606198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:08.606198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:08.606198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:21:08.606198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:08.611053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 202/406] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:21:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:08.618096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96724476-80e7-4ac8-8da2-83e95b5d438c None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:08.649584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:08.703705 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527ffb3-1ac2-472f-a582-11db80785ae0 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:08.711497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:08.742324 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6655b61-0cda-4c01-80cf-d3145bc36aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:08.743139 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6655b61-0cda-4c01-80cf-d3145bc36aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:08.743431 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6655b61-0cda-4c01-80cf-d3145bc36aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:08.743711 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6655b61-0cda-4c01-80cf-d3145bc36aa3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:08.757282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6655b61-0cda-4c01-80cf-d3145bc36aa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:08.766199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-d6655b61-0cda-4c01-80cf-d3145bc36aa3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:21:08.766199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:08.766199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:08.766199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:08.766199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:08.766199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:08.766199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:08.766199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:08.766199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:08.766199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:08.766199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:08.766199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:08.766199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:08.766199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:08.766199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:08.766199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:08.768640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:08.768640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:08.768640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:08.768640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:08.768640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:08.768640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:08.768640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:08.768640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:08.768640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:08.768640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:08.768640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:08.768640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:08.768640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:08.768640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:08.768640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:08.768640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:08.770713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:08.770713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:08.770713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:08.770713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:08.770713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:08.770713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:08.770713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:08.770713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:08.770713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:08.770713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:08.770713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:21:08.770713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:08.770713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 205/407] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:21:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:21:08.777621 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6655b61-0cda-4c01-80cf-d3145bc36aa3 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:08.848592 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96724476-80e7-4ac8-8da2-83e95b5d438c None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:08.918398 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c470b8-9773-416b-921c-973cf890992b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:08.919439 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c470b8-9773-416b-921c-973cf890992b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:08.919930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c470b8-9773-416b-921c-973cf890992b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:08.920409 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c470b8-9773-416b-921c-973cf890992b None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:08.921420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c470b8-9773-416b-921c-973cf890992b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:08.921953 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c470b8-9773-416b-921c-973cf890992b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:08.924321 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c470b8-9773-416b-921c-973cf890992b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:08.940254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 203/408] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:08.946354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c470b8-9773-416b-921c-973cf890992b None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:08.970694 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6655b61-0cda-4c01-80cf-d3145bc36aa3 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:08.985428 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:09.019291 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede37f85-c4ab-4385-a0ad-f17632d1e287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:09.019978 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede37f85-c4ab-4385-a0ad-f17632d1e287 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:09.020273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede37f85-c4ab-4385-a0ad-f17632d1e287 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:09.020560 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede37f85-c4ab-4385-a0ad-f17632d1e287 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:09.021374 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede37f85-c4ab-4385-a0ad-f17632d1e287 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:09.021699 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede37f85-c4ab-4385-a0ad-f17632d1e287 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:09.023687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede37f85-c4ab-4385-a0ad-f17632d1e287 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:09.034597 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 206/409] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:09.036151 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e260a-ae54-4ac5-b17f-35b27ec8e659 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:09.036465 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e260a-ae54-4ac5-b17f-35b27ec8e659 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:09.036752 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e260a-ae54-4ac5-b17f-35b27ec8e659 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:09.038277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 207/410] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:09.044447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e260a-ae54-4ac5-b17f-35b27ec8e659 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:09.113705 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:09.120708 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c470b8-9773-416b-921c-973cf890992b None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:09.158088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016d45a-ccf2-4865-8263-613c04519e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:09.159047 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016d45a-ccf2-4865-8263-613c04519e89 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:09.159408 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016d45a-ccf2-4865-8263-613c04519e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:09.159761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016d45a-ccf2-4865-8263-613c04519e89 None admin] PATH_INFO: `/v3/groups/f15a187fd9ce4df6b26ab36644c35421` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:09.161057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016d45a-ccf2-4865-8263-613c04519e89 None admin] RBAC: Authorizing `identity:get_group(group_id=f15a187fd9ce4df6b26ab36644c35421)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:09.162731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016d45a-ccf2-4865-8263-613c04519e89 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:09.164836 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 204/411] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 17:21:08 2022] GET /identity/v3/groups/f15a187fd9ce4df6b26ab36644c35421 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:09.166383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99dd0db-f0c7-4f6f-a852-8a46e2a37fca None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:09.166771 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99dd0db-f0c7-4f6f-a852-8a46e2a37fca None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:09.167129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99dd0db-f0c7-4f6f-a852-8a46e2a37fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:09.186244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-e99dd0db-f0c7-4f6f-a852-8a46e2a37fca None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:09.187574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-e99dd0db-f0c7-4f6f-a852-8a46e2a37fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:09.189516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e260a-ae54-4ac5-b17f-35b27ec8e659 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:09.248085 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967c6bd-f17b-46e7-b759-06880a255e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:09.249481 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967c6bd-f17b-46e7-b759-06880a255e5c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:09.250000 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967c6bd-f17b-46e7-b759-06880a255e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:09.250792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967c6bd-f17b-46e7-b759-06880a255e5c None admin] PATH_INFO: `/v3/groups/33013e492f804e768fade85df43ac33d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:09.252356 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967c6bd-f17b-46e7-b759-06880a255e5c None admin] RBAC: Authorizing `identity:get_group(group_id=33013e492f804e768fade85df43ac33d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:09.254947 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967c6bd-f17b-46e7-b759-06880a255e5c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:09.257882 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 208/412] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 17:21:09 2022] GET /identity/v3/groups/33013e492f804e768fade85df43ac33d => generated 229 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:09.258887 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967c6bd-f17b-46e7-b759-06880a255e5c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:09.272104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 205/413] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:09.282038 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99dd0db-f0c7-4f6f-a852-8a46e2a37fca None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:09.303226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:09.393646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967c6bd-f17b-46e7-b759-06880a255e5c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:09.401184 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:09.410396 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99dd0db-f0c7-4f6f-a852-8a46e2a37fca None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:09.437548 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc8a99-5ef4-4599-9188-c87c9136b1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:09.438821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc8a99-5ef4-4599-9188-c87c9136b1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:09.439512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc8a99-5ef4-4599-9188-c87c9136b1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:09.440143 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc8a99-5ef4-4599-9188-c87c9136b1c1 None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:09.441594 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc8a99-5ef4-4599-9188-c87c9136b1c1 None admin] RBAC: Authorizing `identity:get_project(project_id=d2415dcdc76d43c486190d67433cf01b)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:09.443277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc8a99-5ef4-4599-9188-c87c9136b1c1 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:09.445727 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 209/414] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:21:09 2022] GET /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:09.446855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc8a99-5ef4-4599-9188-c87c9136b1c1 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:09.447339 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91aef4-9ce9-4a22-9ee3-bdad19be2a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:09.448279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91aef4-9ce9-4a22-9ee3-bdad19be2a7e None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:09.448577 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91aef4-9ce9-4a22-9ee3-bdad19be2a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:09.448860 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91aef4-9ce9-4a22-9ee3-bdad19be2a7e None admin] PATH_INFO: `/v3/projects/26ada52fc9b44bcbbe7f2cc5f37ab303` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:09.450213 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f91aef4-9ce9-4a22-9ee3-bdad19be2a7e None admin] RBAC: Authorizing `identity:get_project(project_id=26ada52fc9b44bcbbe7f2cc5f37ab303)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:09.452402 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f91aef4-9ce9-4a22-9ee3-bdad19be2a7e None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:09.453524 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 206/415] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:21:09 2022] GET /identity/v3/projects/26ada52fc9b44bcbbe7f2cc5f37ab303 => generated 344 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:09.459456 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91aef4-9ce9-4a22-9ee3-bdad19be2a7e None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:09.554763 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc8a99-5ef4-4599-9188-c87c9136b1c1 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:09.565624 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91aef4-9ce9-4a22-9ee3-bdad19be2a7e None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:09.585562 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:09.590804 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73d602-895e-4f79-a0f9-96665b8db1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:09.591840 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73d602-895e-4f79-a0f9-96665b8db1bb None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:09.592474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73d602-895e-4f79-a0f9-96665b8db1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:09.592997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73d602-895e-4f79-a0f9-96665b8db1bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:09.604518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73d602-895e-4f79-a0f9-96665b8db1bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:09.604656 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c83d27-4e5b-4839-840f-8e1692e3c469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:09.605560 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c83d27-4e5b-4839-840f-8e1692e3c469 None admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:09.605850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c83d27-4e5b-4839-840f-8e1692e3c469 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:09.606134 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c83d27-4e5b-4839-840f-8e1692e3c469 None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b/groups/f15a187fd9ce4df6b26ab36644c35421/roles/1423a24997e840608b6c522958481fde` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:09.608384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c83d27-4e5b-4839-840f-8e1692e3c469 None admin] RBAC: Authorizing `identity:create_grant(project_id=d2415dcdc76d43c486190d67433cf01b, group_id=f15a187fd9ce4df6b26ab36644c35421, role_id=1423a24997e840608b6c522958481fde)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:09.609855 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c83d27-4e5b-4839-840f-8e1692e3c469 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:09.615794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-ef73d602-895e-4f79-a0f9-96665b8db1bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:21:09.615794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:09.615794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:09.615794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:09.615794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:09.615794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:09.615794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:09.615794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:09.615794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:09.615794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:09.615794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:09.615794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:09.615794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:09.615794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:09.615794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:09.615794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:09.616718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:09.616718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:09.616718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:09.616718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:09.616718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:09.616718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:09.616718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:09.616718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:09.616718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:09.616718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:09.616718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:09.616718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:09.616718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:09.616718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:09.616718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:09.616718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:09.617650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:09.617650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:09.617650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:09.617650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:09.617650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:09.617650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:09.617650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:09.617650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:09.617650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:09.617650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:09.617650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:21:09.617650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:09.620632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 210/416] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:09.621703 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73d602-895e-4f79-a0f9-96665b8db1bb None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:09.634516 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 207/417] 10.222.0.40 () {58 vars in 1442 bytes} [Thu Jun 16 17:21:09 2022] PUT /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b/groups/f15a187fd9ce4df6b26ab36644c35421/roles/1423a24997e840608b6c522958481fde => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:21:09.635755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c83d27-4e5b-4839-840f-8e1692e3c469 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:09.666591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:09.756978 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73d602-895e-4f79-a0f9-96665b8db1bb None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:09.764151 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c83d27-4e5b-4839-840f-8e1692e3c469 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:09.797712 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe82ccd-602c-4931-8b96-7c0943f49c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:09.798878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe82ccd-602c-4931-8b96-7c0943f49c31 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:09.799615 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe82ccd-602c-4931-8b96-7c0943f49c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:09.799615 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe82ccd-602c-4931-8b96-7c0943f49c31 None admin] PATH_INFO: `/v3/projects/26ada52fc9b44bcbbe7f2cc5f37ab303/groups/33013e492f804e768fade85df43ac33d/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:09.802050 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe82ccd-602c-4931-8b96-7c0943f49c31 None admin] RBAC: Authorizing `identity:create_grant(project_id=26ada52fc9b44bcbbe7f2cc5f37ab303, group_id=33013e492f804e768fade85df43ac33d, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:09.803882 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe82ccd-602c-4931-8b96-7c0943f49c31 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:09.805030 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69e702-9139-46e7-990b-272183d385fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:09.805917 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69e702-9139-46e7-990b-272183d385fa None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:09.806375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69e702-9139-46e7-990b-272183d385fa None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:09.806802 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69e702-9139-46e7-990b-272183d385fa None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:09.807860 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69e702-9139-46e7-990b-272183d385fa None admin] RBAC: Adding query filter params (name=member) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:09.808314 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69e702-9139-46e7-990b-272183d385fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:09.810021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69e702-9139-46e7-990b-272183d385fa None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:09.825360 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 208/418] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:21:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:09.827825 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69e702-9139-46e7-990b-272183d385fa None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:09.837419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 211/419] 10.222.0.40 () {58 vars in 1442 bytes} [Thu Jun 16 17:21:09 2022] PUT /identity/v3/projects/26ada52fc9b44bcbbe7f2cc5f37ab303/groups/33013e492f804e768fade85df43ac33d/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:21:09.954386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:09.976524 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69e702-9139-46e7-990b-272183d385fa None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:10.022979 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca03677-db1b-4816-9501-b492e870dfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:10.023865 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca03677-db1b-4816-9501-b492e870dfbf None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:10.024238 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca03677-db1b-4816-9501-b492e870dfbf None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:10.024525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca03677-db1b-4816-9501-b492e870dfbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:10.047727 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810d70b-4c2f-49cb-ae3d-f40bbfa12f7f None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:10.048024 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810d70b-4c2f-49cb-ae3d-f40bbfa12f7f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:10.048407 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810d70b-4c2f-49cb-ae3d-f40bbfa12f7f None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:10.050521 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 212/420] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:21:10.062815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043bbec-f3af-4c09-870d-63bf98438160 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:10.063104 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043bbec-f3af-4c09-870d-63bf98438160 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:10.063509 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043bbec-f3af-4c09-870d-63bf98438160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:10.094789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1043bbec-f3af-4c09-870d-63bf98438160 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:10.096083 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1043bbec-f3af-4c09-870d-63bf98438160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:10.117836 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:10.126181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca03677-db1b-4816-9501-b492e870dfbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:10.129772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca03677-db1b-4816-9501-b492e870dfbf None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:10.184415 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 213/421] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:10.212761 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043bbec-f3af-4c09-870d-63bf98438160 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:10.222695 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:10.247502 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 209/422] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:21:10.249743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca03677-db1b-4816-9501-b492e870dfbf None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:10.338125 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043bbec-f3af-4c09-870d-63bf98438160 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:10.360310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca03677-db1b-4816-9501-b492e870dfbf None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:10.386704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42dfad-40f3-435d-804d-66dcd42d08ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:10.387936 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42dfad-40f3-435d-804d-66dcd42d08ac None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:10.388431 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42dfad-40f3-435d-804d-66dcd42d08ac None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:10.388945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42dfad-40f3-435d-804d-66dcd42d08ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:10.412676 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42dfad-40f3-435d-804d-66dcd42d08ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:10.426339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-bb42dfad-40f3-435d-804d-66dcd42d08ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:21:10.426339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:10.426339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:10.426339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:10.426339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:10.426339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:10.426339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:10.426339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:10.426339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:10.426339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:10.426339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:10.426339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:10.426339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:10.426339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:10.426339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:10.426339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:10.427342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:10.427342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:10.427342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:10.427342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:10.427342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:10.427342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:10.427342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:10.427342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:10.427342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:10.427342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:10.427342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:10.427342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:10.427342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:10.427342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:10.427342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:10.427342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:10.428186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:10.428186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:10.428186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:10.428186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:10.428186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:10.428186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:10.428186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:10.428186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:10.428186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:10.428186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:10.428186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:21:10.428186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:10.431090 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 214/423] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:21:10.436962 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42dfad-40f3-435d-804d-66dcd42d08ac None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:10.449916 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c6210-99ed-43b6-99f5-d9a0c75bac40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:10.450803 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c6210-99ed-43b6-99f5-d9a0c75bac40 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:10.451094 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c6210-99ed-43b6-99f5-d9a0c75bac40 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:10.451377 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c6210-99ed-43b6-99f5-d9a0c75bac40 None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:10.452664 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c6210-99ed-43b6-99f5-d9a0c75bac40 None admin] RBAC: Authorizing `identity:get_project(project_id=8a3d0e211dd942aaa39eed1730d17e92)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:10.454136 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c6210-99ed-43b6-99f5-d9a0c75bac40 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:10.461041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 210/424] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:21:10 2022] GET /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:10.462451 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a61b4b-ae43-498f-a2af-375ed42e806c None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:10.462752 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a61b4b-ae43-498f-a2af-375ed42e806c None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:10.463044 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a61b4b-ae43-498f-a2af-375ed42e806c None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:10.468467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 211/425] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:10.469324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a61b4b-ae43-498f-a2af-375ed42e806c None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:10.469537 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:10.531291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:10.562660 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42dfad-40f3-435d-804d-66dcd42d08ac None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:10.599111 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a61b4b-ae43-498f-a2af-375ed42e806c None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:10.602337 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96cfd8-7893-4482-a32b-98834e4ec4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:10.603090 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96cfd8-7893-4482-a32b-98834e4ec4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:10.603452 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96cfd8-7893-4482-a32b-98834e4ec4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:10.603830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96cfd8-7893-4482-a32b-98834e4ec4a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:10.604674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96cfd8-7893-4482-a32b-98834e4ec4a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:10.605056 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96cfd8-7893-4482-a32b-98834e4ec4a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:10.606601 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96cfd8-7893-4482-a32b-98834e4ec4a7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:10.616443 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 215/426] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:21:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:10.622351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352faf2-383d-4df1-b47a-fd05ee113f33 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:10.622651 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352faf2-383d-4df1-b47a-fd05ee113f33 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:10.622940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352faf2-383d-4df1-b47a-fd05ee113f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:10.644136 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-2352faf2-383d-4df1-b47a-fd05ee113f33 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:10.645405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2352faf2-383d-4df1-b47a-fd05ee113f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:10.645932 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c425a-f71a-45c9-9a12-f9842f40ad32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:10.646806 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c425a-f71a-45c9-9a12-f9842f40ad32 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:10.647148 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c425a-f71a-45c9-9a12-f9842f40ad32 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:10.647729 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c425a-f71a-45c9-9a12-f9842f40ad32 None admin] PATH_INFO: `/v3/groups/f15a187fd9ce4df6b26ab36644c35421` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:10.649967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c425a-f71a-45c9-9a12-f9842f40ad32 None admin] RBAC: Authorizing `identity:get_group(group_id=f15a187fd9ce4df6b26ab36644c35421)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:10.651481 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c425a-f71a-45c9-9a12-f9842f40ad32 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:10.653203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 212/427] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 17:21:10 2022] GET /identity/v3/groups/f15a187fd9ce4df6b26ab36644c35421 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:10.654623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c425a-f71a-45c9-9a12-f9842f40ad32 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:10.718757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:10.725020 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 216/428] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:10.726000 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352faf2-383d-4df1-b47a-fd05ee113f33 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:10.799440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c425a-f71a-45c9-9a12-f9842f40ad32 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:10.822262 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:10.837807 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995e7a8-c651-4dbd-a96d-5a08ff2b6cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:10.838597 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995e7a8-c651-4dbd-a96d-5a08ff2b6cae None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:10.838953 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995e7a8-c651-4dbd-a96d-5a08ff2b6cae None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:10.839332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995e7a8-c651-4dbd-a96d-5a08ff2b6cae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:10.856532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352faf2-383d-4df1-b47a-fd05ee113f33 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:10.912846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a81e81-3ed1-4c9b-b6f3-094e0704a0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:10.914038 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a81e81-3ed1-4c9b-b6f3-094e0704a0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:10.914498 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a81e81-3ed1-4c9b-b6f3-094e0704a0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:10.914940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a81e81-3ed1-4c9b-b6f3-094e0704a0a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:10.916194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a81e81-3ed1-4c9b-b6f3-094e0704a0a7 None admin] RBAC: Adding query filter params (group.id=f15a187fd9ce4df6b26ab36644c35421, role.id=483146fdc4ac47cf97c04a9f66c091b4, scope.project.id=8a3d0e211dd942aaa39eed1730d17e92) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:10.916665 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a81e81-3ed1-4c9b-b6f3-094e0704a0a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:10.919184 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a81e81-3ed1-4c9b-b6f3-094e0704a0a7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:10.919640 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995e7a8-c651-4dbd-a96d-5a08ff2b6cae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:10.921217 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995e7a8-c651-4dbd-a96d-5a08ff2b6cae None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:10.928371 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 217/429] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jun 16 17:21:10 2022] GET /identity/v3/role_assignments?group.id=f15a187fd9ce4df6b26ab36644c35421&scope.project.id=8a3d0e211dd942aaa39eed1730d17e92&role.id=483146fdc4ac47cf97c04a9f66c091b4 => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:10.929468 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a81e81-3ed1-4c9b-b6f3-094e0704a0a7 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:11.027069 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:11.085524 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a81e81-3ed1-4c9b-b6f3-094e0704a0a7 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:11.125276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 213/430] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:21:11.132208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995e7a8-c651-4dbd-a96d-5a08ff2b6cae None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:11.132470 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfabeb6-51e4-4157-8faf-e87a607e78f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:11.133337 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfabeb6-51e4-4157-8faf-e87a607e78f3 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:11.133632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfabeb6-51e4-4157-8faf-e87a607e78f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:11.133918 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfabeb6-51e4-4157-8faf-e87a607e78f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:11.149310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfabeb6-51e4-4157-8faf-e87a607e78f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:11.160874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-2dfabeb6-51e4-4157-8faf-e87a607e78f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:21:11.160874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:11.160874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:11.160874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:11.160874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:11.160874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:11.160874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:11.160874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:11.160874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:11.160874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:11.160874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:11.160874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:11.160874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:11.160874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:11.160874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:11.160874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:11.161833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:11.161833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:11.161833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:11.161833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:11.161833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:11.161833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:11.161833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:11.161833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:11.161833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:11.161833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:11.161833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:11.161833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:11.161833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:11.161833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:11.161833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:11.161833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:11.162666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:11.162666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:11.162666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:11.162666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:11.162666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:11.162666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:11.162666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:11.162666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:11.162666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:11.162666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:11.162666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:21:11.162666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:11.162666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 218/431] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:21:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:11.167908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfabeb6-51e4-4157-8faf-e87a607e78f3 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:11.169973 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:11.263141 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995e7a8-c651-4dbd-a96d-5a08ff2b6cae None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:11.285213 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfabeb6-51e4-4157-8faf-e87a607e78f3 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:11.302715 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00363d8-55a4-41a3-aa3a-9224efc9bf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:11.303850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00363d8-55a4-41a3-aa3a-9224efc9bf9d None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:11.303980 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00363d8-55a4-41a3-aa3a-9224efc9bf9d None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:11.304250 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00363d8-55a4-41a3-aa3a-9224efc9bf9d None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:11.305506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00363d8-55a4-41a3-aa3a-9224efc9bf9d None admin] RBAC: Authorizing `identity:get_project(project_id=d2415dcdc76d43c486190d67433cf01b)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:11.306993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00363d8-55a4-41a3-aa3a-9224efc9bf9d None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:11.309140 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 214/432] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:21:11 2022] GET /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:11.314605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:11.316485 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00363d8-55a4-41a3-aa3a-9224efc9bf9d None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:11.323457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb4573-d0c5-4232-b649-ed964d77590c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:11.324568 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb4573-d0c5-4232-b649-ed964d77590c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:11.325622 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb4573-d0c5-4232-b649-ed964d77590c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:11.325622 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb4573-d0c5-4232-b649-ed964d77590c None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:11.326136 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb4573-d0c5-4232-b649-ed964d77590c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:11.326524 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb4573-d0c5-4232-b649-ed964d77590c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:11.327996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb4573-d0c5-4232-b649-ed964d77590c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:11.337820 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 219/433] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:11.343415 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb4573-d0c5-4232-b649-ed964d77590c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:11.422013 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:11.439293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00363d8-55a4-41a3-aa3a-9224efc9bf9d None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:11.457116 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb4573-d0c5-4232-b649-ed964d77590c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:11.478931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93774eb5-882a-4590-92cb-068a99938c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:11.480567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93774eb5-882a-4590-92cb-068a99938c45 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:11.480862 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93774eb5-882a-4590-92cb-068a99938c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:11.481166 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93774eb5-882a-4590-92cb-068a99938c45 None admin] PATH_INFO: `/v3/groups/f15a187fd9ce4df6b26ab36644c35421` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:11.484562 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93774eb5-882a-4590-92cb-068a99938c45 None admin] RBAC: Authorizing `identity:get_group(group_id=f15a187fd9ce4df6b26ab36644c35421)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:11.486217 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93774eb5-882a-4590-92cb-068a99938c45 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:11.493337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 215/434] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 17:21:11 2022] GET /identity/v3/groups/f15a187fd9ce4df6b26ab36644c35421 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:11.495175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93774eb5-882a-4590-92cb-068a99938c45 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:11.502848 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bd6d4-9640-4efc-88b7-2e85fdbdc69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:11.503781 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bd6d4-9640-4efc-88b7-2e85fdbdc69b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:11.504261 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bd6d4-9640-4efc-88b7-2e85fdbdc69b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:11.504757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bd6d4-9640-4efc-88b7-2e85fdbdc69b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:11.562651 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:11.577015 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bd6d4-9640-4efc-88b7-2e85fdbdc69b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:11.578926 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bd6d4-9640-4efc-88b7-2e85fdbdc69b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:11.614300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93774eb5-882a-4590-92cb-068a99938c45 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:11.656299 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe173c87-b74c-4da2-a31a-a2fd8592914f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:11.656983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe173c87-b74c-4da2-a31a-a2fd8592914f None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:11.657311 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe173c87-b74c-4da2-a31a-a2fd8592914f None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:11.658141 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe173c87-b74c-4da2-a31a-a2fd8592914f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:11.659074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe173c87-b74c-4da2-a31a-a2fd8592914f None admin] RBAC: Adding query filter params (group.id=f15a187fd9ce4df6b26ab36644c35421, role.id=483146fdc4ac47cf97c04a9f66c091b4, scope.project.id=d2415dcdc76d43c486190d67433cf01b) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:11.659428 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe173c87-b74c-4da2-a31a-a2fd8592914f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:11.661571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe173c87-b74c-4da2-a31a-a2fd8592914f None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:11.673418 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 216/435] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jun 16 17:21:11 2022] GET /identity/v3/role_assignments?group.id=f15a187fd9ce4df6b26ab36644c35421&scope.project.id=d2415dcdc76d43c486190d67433cf01b&role.id=483146fdc4ac47cf97c04a9f66c091b4 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:11.704063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 220/436] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:21:11.711500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe173c87-b74c-4da2-a31a-a2fd8592914f None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:11.742315 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:11.839251 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe173c87-b74c-4da2-a31a-a2fd8592914f None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:11.879314 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba9d1e-70ca-4418-a7c2-a0961aec8651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:11.880193 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba9d1e-70ca-4418-a7c2-a0961aec8651 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:11.880482 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba9d1e-70ca-4418-a7c2-a0961aec8651 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:11.880767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba9d1e-70ca-4418-a7c2-a0961aec8651 None admin] PATH_INFO: `/v3/projects/8a3d0e211dd942aaa39eed1730d17e92` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:11.882006 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba9d1e-70ca-4418-a7c2-a0961aec8651 None admin] RBAC: Authorizing `identity:get_project(project_id=8a3d0e211dd942aaa39eed1730d17e92)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:11.883446 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba9d1e-70ca-4418-a7c2-a0961aec8651 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:11.885619 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 217/437] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:21:11 2022] GET /identity/v3/projects/8a3d0e211dd942aaa39eed1730d17e92 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:11.891074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bd6d4-9640-4efc-88b7-2e85fdbdc69b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:11.992869 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:12.008596 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bd6d4-9640-4efc-88b7-2e85fdbdc69b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:12.048266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b863dc-515e-4a45-be9d-2f94c0b468f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:12.049456 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b863dc-515e-4a45-be9d-2f94c0b468f0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:12.049747 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b863dc-515e-4a45-be9d-2f94c0b468f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:12.050027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b863dc-515e-4a45-be9d-2f94c0b468f0 None admin] PATH_INFO: `/v3/groups/f15a187fd9ce4df6b26ab36644c35421` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:12.051775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b863dc-515e-4a45-be9d-2f94c0b468f0 None admin] RBAC: Authorizing `identity:get_group(group_id=f15a187fd9ce4df6b26ab36644c35421)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:12.053261 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b863dc-515e-4a45-be9d-2f94c0b468f0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:12.059285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 221/438] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 17:21:11 2022] GET /identity/v3/groups/f15a187fd9ce4df6b26ab36644c35421 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:12.061351 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba9d1e-70ca-4418-a7c2-a0961aec8651 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:12.168678 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:12.182612 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba9d1e-70ca-4418-a7c2-a0961aec8651 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:12.202078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc48c0-6527-478f-b634-8a4db8db2db2 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:12.202391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc48c0-6527-478f-b634-8a4db8db2db2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:12.202640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc48c0-6527-478f-b634-8a4db8db2db2 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:12.204171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 222/439] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:12.209820 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf4cbc-15bd-4b6d-9f7a-5643baa2d2ff None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:12.210117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf4cbc-15bd-4b6d-9f7a-5643baa2d2ff None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:12.210399 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf4cbc-15bd-4b6d-9f7a-5643baa2d2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:12.221926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5e750-9389-4e0b-b84a-84eee50f4714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:12.222841 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5e750-9389-4e0b-b84a-84eee50f4714 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:12.223294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5e750-9389-4e0b-b84a-84eee50f4714 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:12.223730 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5e750-9389-4e0b-b84a-84eee50f4714 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:12.224726 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c5e750-9389-4e0b-b84a-84eee50f4714 None admin] RBAC: Adding query filter params (group.id=f15a187fd9ce4df6b26ab36644c35421, role.id=1423a24997e840608b6c522958481fde, scope.project.id=8a3d0e211dd942aaa39eed1730d17e92) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:12.225222 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c5e750-9389-4e0b-b84a-84eee50f4714 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:12.226808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c5e750-9389-4e0b-b84a-84eee50f4714 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:12.227305 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-8daf4cbc-15bd-4b6d-9f7a-5643baa2d2ff None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:12.228505 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8daf4cbc-15bd-4b6d-9f7a-5643baa2d2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:12.239966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 218/440] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jun 16 17:21:12 2022] GET /identity/v3/role_assignments?group.id=f15a187fd9ce4df6b26ab36644c35421&scope.project.id=8a3d0e211dd942aaa39eed1730d17e92&role.id=1423a24997e840608b6c522958481fde => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:12.313465 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 223/441] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:21:12.342051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5e750-9389-4e0b-b84a-84eee50f4714 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:12.459147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5e750-9389-4e0b-b84a-84eee50f4714 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:12.494335 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd67e4-30c9-443b-8e67-0089b1c13dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:12.495165 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd67e4-30c9-443b-8e67-0089b1c13dda None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:12.495452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd67e4-30c9-443b-8e67-0089b1c13dda None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:12.495726 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd67e4-30c9-443b-8e67-0089b1c13dda None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:12.508979 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd67e4-30c9-443b-8e67-0089b1c13dda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:12.518042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-20cd67e4-30c9-443b-8e67-0089b1c13dda None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:21:12.518042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:12.518042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:12.518042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:12.518042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:12.518042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:12.518042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:12.518042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:12.518042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:12.518042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:12.518042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:12.518042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:12.518042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:12.518042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:12.518042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:12.518042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:12.519082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:12.519082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:12.519082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:12.519082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:12.519082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:12.519082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:12.519082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:12.519082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:12.519082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:12.519082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:12.519082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:12.519082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:12.519082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:12.519082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:12.519082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:12.519082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:12.520362 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:12.520362 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:12.520362 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:12.520362 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:12.520362 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:12.520362 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:12.520362 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:12.520362 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:12.520362 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:12.520362 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:12.520362 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:21:12.520362 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:12.520362 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 219/442] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:21:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:12.525755 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf4cbc-15bd-4b6d-9f7a-5643baa2d2ff None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:12.575793 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cc115-565e-4cb3-a345-d16321bd11eb None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:12.576076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cc115-565e-4cb3-a345-d16321bd11eb None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:12.576295 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cc115-565e-4cb3-a345-d16321bd11eb None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:12.577690 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 220/443] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:12.582778 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f73ca-8d2f-4070-bf7c-24063d49403b None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:12.583072 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f73ca-8d2f-4070-bf7c-24063d49403b None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:12.583352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f73ca-8d2f-4070-bf7c-24063d49403b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:12.604214 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-636f73ca-8d2f-4070-bf7c-24063d49403b None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:12.605619 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-636f73ca-8d2f-4070-bf7c-24063d49403b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:12.614742 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:12.627268 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:12.646568 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf4cbc-15bd-4b6d-9f7a-5643baa2d2ff None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:12.687000 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c38402-0205-4d25-9a51-51f863840cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:12.687850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c38402-0205-4d25-9a51-51f863840cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:12.688135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c38402-0205-4d25-9a51-51f863840cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:12.688429 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c38402-0205-4d25-9a51-51f863840cf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:12.689970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c38402-0205-4d25-9a51-51f863840cf6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:12.689970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c38402-0205-4d25-9a51-51f863840cf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:12.691302 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c38402-0205-4d25-9a51-51f863840cf6 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:12.696573 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 221/444] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:12.704215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 224/445] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:12.710492 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f73ca-8d2f-4070-bf7c-24063d49403b None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:12.727161 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c38402-0205-4d25-9a51-51f863840cf6 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:12.844533 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c38402-0205-4d25-9a51-51f863840cf6 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:12.851924 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f73ca-8d2f-4070-bf7c-24063d49403b None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:12.878138 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12ddae-b6b8-49f3-b2dd-cb955a9dae22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:12.878935 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12ddae-b6b8-49f3-b2dd-cb955a9dae22 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:12.879221 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12ddae-b6b8-49f3-b2dd-cb955a9dae22 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:12.879500 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12ddae-b6b8-49f3-b2dd-cb955a9dae22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:12.886894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46b7ec-9fe8-4393-b06f-09bb0eb8ffe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:12.887545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46b7ec-9fe8-4393-b06f-09bb0eb8ffe9 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:12.887836 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46b7ec-9fe8-4393-b06f-09bb0eb8ffe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:12.888115 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46b7ec-9fe8-4393-b06f-09bb0eb8ffe9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:12.890979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee12ddae-b6b8-49f3-b2dd-cb955a9dae22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:12.899380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-ee12ddae-b6b8-49f3-b2dd-cb955a9dae22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:21:12.899380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:12.899380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:12.899380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:12.899380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:12.899380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:12.899380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:12.899380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:12.899380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:12.899380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:12.899380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:12.899380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:12.899380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:12.899380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:12.899380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:12.899380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:12.900514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:12.900514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:12.900514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:12.900514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:12.900514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:12.900514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:12.900514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:12.900514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:12.900514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:12.900514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:12.900514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:12.900514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:12.900514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:12.900514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:12.900514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:12.900514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:12.901425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:12.901425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:12.901425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:12.901425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:12.901425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:12.901425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:12.901425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:12.901425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:12.901425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:12.901425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:12.901425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:21:12.901425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:12.902073 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:12.902143 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 225/446] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:21:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:21:12.906227 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12ddae-b6b8-49f3-b2dd-cb955a9dae22 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:12.915666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:12.960497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46b7ec-9fe8-4393-b06f-09bb0eb8ffe9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:12.962015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46b7ec-9fe8-4393-b06f-09bb0eb8ffe9 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:13.006556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12ddae-b6b8-49f3-b2dd-cb955a9dae22 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:13.039977 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266f2e5-16cd-4543-9e0d-13b37f62c21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:13.040646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266f2e5-16cd-4543-9e0d-13b37f62c21f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:13.040938 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266f2e5-16cd-4543-9e0d-13b37f62c21f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:13.041255 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266f2e5-16cd-4543-9e0d-13b37f62c21f None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:13.042010 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266f2e5-16cd-4543-9e0d-13b37f62c21f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:13.042321 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266f2e5-16cd-4543-9e0d-13b37f62c21f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:13.043689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266f2e5-16cd-4543-9e0d-13b37f62c21f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:13.052738 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 226/447] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:13.058162 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266f2e5-16cd-4543-9e0d-13b37f62c21f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:13.062382 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 222/448] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:21:13.068478 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46b7ec-9fe8-4393-b06f-09bb0eb8ffe9 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:13.114460 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:13.133204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:13.180184 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266f2e5-16cd-4543-9e0d-13b37f62c21f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:13.186154 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46b7ec-9fe8-4393-b06f-09bb0eb8ffe9 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:13.229491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36b304-5e14-4ecd-a96f-c830c97b9b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:13.230465 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bf97c-f51f-4ffe-af4f-d47aa4bd7b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:13.230592 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36b304-5e14-4ecd-a96f-c830c97b9b6f None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:13.230856 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36b304-5e14-4ecd-a96f-c830c97b9b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:13.231198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36b304-5e14-4ecd-a96f-c830c97b9b6f None admin] PATH_INFO: `/v3/projects/d2415dcdc76d43c486190d67433cf01b` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:13.231451 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bf97c-f51f-4ffe-af4f-d47aa4bd7b1a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:13.231751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bf97c-f51f-4ffe-af4f-d47aa4bd7b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:13.232041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bf97c-f51f-4ffe-af4f-d47aa4bd7b1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:13.233189 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36b304-5e14-4ecd-a96f-c830c97b9b6f None admin] RBAC: Authorizing `identity:get_project(project_id=d2415dcdc76d43c486190d67433cf01b)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:13.234444 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36b304-5e14-4ecd-a96f-c830c97b9b6f None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:13.237319 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 223/449] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:21:13 2022] GET /identity/v3/projects/d2415dcdc76d43c486190d67433cf01b => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:13.242721 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36b304-5e14-4ecd-a96f-c830c97b9b6f None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:13.310347 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387bf97c-f51f-4ffe-af4f-d47aa4bd7b1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:13.312145 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387bf97c-f51f-4ffe-af4f-d47aa4bd7b1a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:13.344616 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36b304-5e14-4ecd-a96f-c830c97b9b6f None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:13.355971 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:13.384450 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:13.384627 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc8d03-2bd9-4561-8737-01a457b305a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:13.385441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc8d03-2bd9-4561-8737-01a457b305a8 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:13.385579 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc8d03-2bd9-4561-8737-01a457b305a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:13.385873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc8d03-2bd9-4561-8737-01a457b305a8 None admin] PATH_INFO: `/v3/groups/f15a187fd9ce4df6b26ab36644c35421` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:13.387308 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc8d03-2bd9-4561-8737-01a457b305a8 None admin] RBAC: Authorizing `identity:get_group(group_id=f15a187fd9ce4df6b26ab36644c35421)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:13.388747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc8d03-2bd9-4561-8737-01a457b305a8 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:13.390637 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 224/450] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 17:21:13 2022] GET /identity/v3/groups/f15a187fd9ce4df6b26ab36644c35421 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:13.397233 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc8d03-2bd9-4561-8737-01a457b305a8 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:13.431689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 227/451] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:21:13.437263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bf97c-f51f-4ffe-af4f-d47aa4bd7b1a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:13.504385 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc8d03-2bd9-4561-8737-01a457b305a8 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:13.537698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bf97c-f51f-4ffe-af4f-d47aa4bd7b1a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:13.538894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5c0da-dfff-4d00-823d-43e3b940e116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:13.539550 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5c0da-dfff-4d00-823d-43e3b940e116 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:13.539902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5c0da-dfff-4d00-823d-43e3b940e116 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:13.540185 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5c0da-dfff-4d00-823d-43e3b940e116 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:13.541014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5c0da-dfff-4d00-823d-43e3b940e116 None admin] RBAC: Adding query filter params (group.id=f15a187fd9ce4df6b26ab36644c35421, role.id=1423a24997e840608b6c522958481fde, scope.project.id=d2415dcdc76d43c486190d67433cf01b) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:13.541356 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5c0da-dfff-4d00-823d-43e3b940e116 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:13.542799 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5c0da-dfff-4d00-823d-43e3b940e116 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:13.548619 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 225/452] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jun 16 17:21:13 2022] GET /identity/v3/role_assignments?group.id=f15a187fd9ce4df6b26ab36644c35421&scope.project.id=d2415dcdc76d43c486190d67433cf01b&role.id=1423a24997e840608b6c522958481fde => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:13.573884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d367358-0dd0-4e51-af90-b975376c53bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:13.574776 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d367358-0dd0-4e51-af90-b975376c53bf None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:13.575064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d367358-0dd0-4e51-af90-b975376c53bf None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:13.575347 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d367358-0dd0-4e51-af90-b975376c53bf None admin] PATH_INFO: `/v3/projects/26ada52fc9b44bcbbe7f2cc5f37ab303` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:13.576661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d367358-0dd0-4e51-af90-b975376c53bf None admin] RBAC: Authorizing `identity:get_project(project_id=26ada52fc9b44bcbbe7f2cc5f37ab303)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:13.578192 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d367358-0dd0-4e51-af90-b975376c53bf None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:13.580571 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 228/453] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:21:13 2022] GET /identity/v3/projects/26ada52fc9b44bcbbe7f2cc5f37ab303 => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:13.586398 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5c0da-dfff-4d00-823d-43e3b940e116 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:13.632632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:13.694861 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5c0da-dfff-4d00-823d-43e3b940e116 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:13.734534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4dd28f-6e57-44f7-943f-8f128fb50c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:13.735427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dd28f-6e57-44f7-943f-8f128fb50c69 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:13.736456 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dd28f-6e57-44f7-943f-8f128fb50c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:13.736549 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dd28f-6e57-44f7-943f-8f128fb50c69 None admin] PATH_INFO: `/v3/groups/33013e492f804e768fade85df43ac33d` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:13.737759 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4dd28f-6e57-44f7-943f-8f128fb50c69 None admin] RBAC: Authorizing `identity:get_group(group_id=33013e492f804e768fade85df43ac33d)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:13.739491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4dd28f-6e57-44f7-943f-8f128fb50c69 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:13.740837 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 226/454] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 17:21:13 2022] GET /identity/v3/groups/33013e492f804e768fade85df43ac33d => generated 229 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:13.747173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d367358-0dd0-4e51-af90-b975376c53bf None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:13.784074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:13.851488 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d367358-0dd0-4e51-af90-b975376c53bf None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:13.886867 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840e04e-9715-42d0-9a93-1283b38eb027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:13.887529 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840e04e-9715-42d0-9a93-1283b38eb027 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:13.887817 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840e04e-9715-42d0-9a93-1283b38eb027 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:13.888099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840e04e-9715-42d0-9a93-1283b38eb027 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:13.888943 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840e04e-9715-42d0-9a93-1283b38eb027 None admin] RBAC: Adding query filter params (group.id=33013e492f804e768fade85df43ac33d, role.id=25a16d8ea4084e9eacdebb7a4dcd2337, scope.project.id=26ada52fc9b44bcbbe7f2cc5f37ab303) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:13.889290 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840e04e-9715-42d0-9a93-1283b38eb027 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:13.890728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840e04e-9715-42d0-9a93-1283b38eb027 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:13.896999 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 229/455] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jun 16 17:21:13 2022] GET /identity/v3/role_assignments?group.id=33013e492f804e768fade85df43ac33d&scope.project.id=26ada52fc9b44bcbbe7f2cc5f37ab303&role.id=25a16d8ea4084e9eacdebb7a4dcd2337 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:16.443605 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb1f3f-5a34-4716-a863-0832bce8aef0 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:16.444459 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb1f3f-5a34-4716-a863-0832bce8aef0 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:16.444925 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb1f3f-5a34-4716-a863-0832bce8aef0 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:16.446716 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 227/456] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:16.452157 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6ace3-cce1-411e-8d58-927750f55b6b None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:16.452907 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6ace3-cce1-411e-8d58-927750f55b6b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:16.453372 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6ace3-cce1-411e-8d58-927750f55b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:16.482403 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-efe6ace3-cce1-411e-8d58-927750f55b6b None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:16.483852 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-efe6ace3-cce1-411e-8d58-927750f55b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:16.583066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 230/457] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:21:16.632314 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb1f3f-5a34-4716-a863-0832bce8aef0 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:16.718265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95734633-43bb-4a73-a2c1-333ddacee4c0 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:16.718936 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95734633-43bb-4a73-a2c1-333ddacee4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:16.719427 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95734633-43bb-4a73-a2c1-333ddacee4c0 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:16.722225 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 231/458] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:16.739354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e4592-06e6-4147-aa03-9886f8d4a211 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:16.739641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e4592-06e6-4147-aa03-9886f8d4a211 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:16.740110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e4592-06e6-4147-aa03-9886f8d4a211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:16.772198 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-282e4592-06e6-4147-aa03-9886f8d4a211 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:16.772198 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-282e4592-06e6-4147-aa03-9886f8d4a211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:16.772847 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb1f3f-5a34-4716-a863-0832bce8aef0 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:16.791415 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:16.822421 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bd9f2-077e-42be-9aba-d27ccff1cedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:16.823234 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bd9f2-077e-42be-9aba-d27ccff1cedc None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:16.823527 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bd9f2-077e-42be-9aba-d27ccff1cedc None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:16.823811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bd9f2-077e-42be-9aba-d27ccff1cedc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:16.826908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:16.846811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bd9f2-077e-42be-9aba-d27ccff1cedc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:16.848208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bd9f2-077e-42be-9aba-d27ccff1cedc None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:16.887663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-f84bd9f2-077e-42be-9aba-d27ccff1cedc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:16.887663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:16.887663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:16.887663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:16.887663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:16.887663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:16.887663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:16.887663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:16.887663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:16.887663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:16.887663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:16.887663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:16.887663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:16.887663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:16.887663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:16.888829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:16.888829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:16.888829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:16.888829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:16.888829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:16.888829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:16.888829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:16.888829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:16.888829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:16.888829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:16.888829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:16.888829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:16.888829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:16.888829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:16.888829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:16.888829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:16.888829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:16.890220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:16.890220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:16.890220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:16.890220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:16.890220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 228/459] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:16.896186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bd9f2-077e-42be-9aba-d27ccff1cedc None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:16.916064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 232/460] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:16.942782 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e4592-06e6-4147-aa03-9886f8d4a211 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:17.078329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bd9f2-077e-42be-9aba-d27ccff1cedc None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:17.117565 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e4592-06e6-4147-aa03-9886f8d4a211 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:17.131517 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f28d20-29c0-43df-8fd7-8342b1dbd943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:17.132470 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f28d20-29c0-43df-8fd7-8342b1dbd943 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:17.132957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f28d20-29c0-43df-8fd7-8342b1dbd943 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:17.133670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f28d20-29c0-43df-8fd7-8342b1dbd943 None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:17.135306 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f28d20-29c0-43df-8fd7-8342b1dbd943 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:17.136053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f28d20-29c0-43df-8fd7-8342b1dbd943 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:17.138240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f28d20-29c0-43df-8fd7-8342b1dbd943 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:17.156573 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7e63a-01b6-42ac-b35f-211a633b6f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:17.156800 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 229/461] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:17.157878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7e63a-01b6-42ac-b35f-211a633b6f21 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:17.158312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7e63a-01b6-42ac-b35f-211a633b6f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:17.158740 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7e63a-01b6-42ac-b35f-211a633b6f21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:17.162161 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462f6bb-520e-46b9-9a73-2a4c04a04cda None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:17.162605 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462f6bb-520e-46b9-9a73-2a4c04a04cda None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:17.163766 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462f6bb-520e-46b9-9a73-2a4c04a04cda None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:17.171506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 230/462] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:17 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:17.171506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462f6bb-520e-46b9-9a73-2a4c04a04cda None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:17.176645 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7e63a-01b6-42ac-b35f-211a633b6f21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:17.178386 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7e63a-01b6-42ac-b35f-211a633b6f21 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:17.197915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-dec7e63a-01b6-42ac-b35f-211a633b6f21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:17.197915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:17.197915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:17.197915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:17.197915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:17.197915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:17.197915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:17.197915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:17.197915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:17.197915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:17.197915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:17.197915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:17.197915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:17.197915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:17.197915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:17.198811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:17.198811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:17.198811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:17.198811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:17.198811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:17.198811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:17.198811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:17.198811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:17.198811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:17.198811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:17.198811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:17.198811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:17.198811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:17.198811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:17.198811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:17.198811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:17.198811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:17.199761 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:17.199761 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:17.199761 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:17.199761 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:17.200785 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:17.203129 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 233/463] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:21:17.205182 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5691b-30d2-45fa-b381-ead900e4b6b2 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:17.205683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5691b-30d2-45fa-b381-ead900e4b6b2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:17.206177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5691b-30d2-45fa-b381-ead900e4b6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:17.226704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-cda5691b-30d2-45fa-b381-ead900e4b6b2 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:17.228302 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-cda5691b-30d2-45fa-b381-ead900e4b6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:17.238837 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:17.315045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 234/464] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:17.318115 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5691b-30d2-45fa-b381-ead900e4b6b2 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:17.319834 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462f6bb-520e-46b9-9a73-2a4c04a04cda None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:17.357000 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9ea93-82ab-4195-ae3e-e9b11d598d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:17.357709 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9ea93-82ab-4195-ae3e-e9b11d598d7b None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:17.358143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9ea93-82ab-4195-ae3e-e9b11d598d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:17.358310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9ea93-82ab-4195-ae3e-e9b11d598d7b None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:17.358972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9ea93-82ab-4195-ae3e-e9b11d598d7b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:17.360440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9ea93-82ab-4195-ae3e-e9b11d598d7b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:17.390147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 231/465] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 17:21:17 2022] POST /identity/v3/users => generated 246 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:21:17.390147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6089b-d8ef-46a7-969f-5cd5fa777c9f None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:17.390147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6089b-d8ef-46a7-969f-5cd5fa777c9f None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:17.390147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6089b-d8ef-46a7-969f-5cd5fa777c9f None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:17.390147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 232/466] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:17.390147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6089b-d8ef-46a7-969f-5cd5fa777c9f None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:17.469273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5691b-30d2-45fa-b381-ead900e4b6b2 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:17.495398 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:17.523804 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b6bb0-dc64-4bc5-b3a9-8850e9208746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:17.525217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b6bb0-dc64-4bc5-b3a9-8850e9208746 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:17.526304 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b6bb0-dc64-4bc5-b3a9-8850e9208746 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:17.527102 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b6bb0-dc64-4bc5-b3a9-8850e9208746 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:17.528078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b6bb0-dc64-4bc5-b3a9-8850e9208746 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:17.528600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b6bb0-dc64-4bc5-b3a9-8850e9208746 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:17.529709 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6089b-d8ef-46a7-969f-5cd5fa777c9f None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:17.530588 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b6bb0-dc64-4bc5-b3a9-8850e9208746 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:17.549891 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 235/467] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:17.552282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741dbc29-0237-4114-936d-6150b9d4f4b6 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:17.552788 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741dbc29-0237-4114-936d-6150b9d4f4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:17.553307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741dbc29-0237-4114-936d-6150b9d4f4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:17.560169 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:17.577795 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-741dbc29-0237-4114-936d-6150b9d4f4b6 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:17.579025 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-741dbc29-0237-4114-936d-6150b9d4f4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:17.582288 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d6cc6-5264-48f5-8000-deacc3846c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:17.583412 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d6cc6-5264-48f5-8000-deacc3846c27 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:17.583937 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d6cc6-5264-48f5-8000-deacc3846c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:17.584425 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d6cc6-5264-48f5-8000-deacc3846c27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:17.598837 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d6cc6-5264-48f5-8000-deacc3846c27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:17.600375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d6cc6-5264-48f5-8000-deacc3846c27 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:17.623891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-243d6cc6-5264-48f5-8000-deacc3846c27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:17.623891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:17.623891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:17.623891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:17.623891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:17.623891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:17.623891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:17.623891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:17.623891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:17.623891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:17.623891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:17.623891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:17.623891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:17.623891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:17.623891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:17.624842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:17.624842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:17.624842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:17.624842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:17.624842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:17.624842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:17.624842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:17.624842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:17.624842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:17.624842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:17.624842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:17.624842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:17.624842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:17.624842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:17.624842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:17.624842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:17.624842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:17.625798 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:17.625798 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:17.625798 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:17.625798 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:17.628748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 233/468] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:17.629772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d6cc6-5264-48f5-8000-deacc3846c27 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:17.672186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 236/469] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:17.674543 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741dbc29-0237-4114-936d-6150b9d4f4b6 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:17.762212 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d6cc6-5264-48f5-8000-deacc3846c27 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:17.794456 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:17.801605 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26814a8-d672-4165-ba72-e3a1a1420421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:17.802287 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26814a8-d672-4165-ba72-e3a1a1420421 None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:17.802574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26814a8-d672-4165-ba72-e3a1a1420421 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:17.802855 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26814a8-d672-4165-ba72-e3a1a1420421 None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:17.803577 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26814a8-d672-4165-ba72-e3a1a1420421 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:17.805028 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26814a8-d672-4165-ba72-e3a1a1420421 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:17.814641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741dbc29-0237-4114-936d-6150b9d4f4b6 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:17.828407 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 234/470] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 17:21:17 2022] POST /identity/v3/users => generated 248 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:21:17.830286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26814a8-d672-4165-ba72-e3a1a1420421 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:17.861176 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:17.868479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae92b88-ab49-4bbd-8360-165893ae31d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:17.869622 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae92b88-ab49-4bbd-8360-165893ae31d7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:17.870115 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae92b88-ab49-4bbd-8360-165893ae31d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:17.870660 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae92b88-ab49-4bbd-8360-165893ae31d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:17.871605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae92b88-ab49-4bbd-8360-165893ae31d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:17.872117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae92b88-ab49-4bbd-8360-165893ae31d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:17.873835 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae92b88-ab49-4bbd-8360-165893ae31d7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:17.897231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 237/471] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:17.904285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae92b88-ab49-4bbd-8360-165893ae31d7 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:17.950637 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26814a8-d672-4165-ba72-e3a1a1420421 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:17.996253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7176393-d00b-4eaf-8ade-88e273e45815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:17.997077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7176393-d00b-4eaf-8ade-88e273e45815 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:17.997388 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7176393-d00b-4eaf-8ade-88e273e45815 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:17.997665 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7176393-d00b-4eaf-8ade-88e273e45815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:18.024029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7176393-d00b-4eaf-8ade-88e273e45815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:18.025549 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7176393-d00b-4eaf-8ade-88e273e45815 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:18.051525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-a7176393-d00b-4eaf-8ade-88e273e45815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:18.051525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:18.051525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:18.051525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:18.051525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:18.051525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:18.051525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:18.051525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:18.051525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:18.051525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:18.051525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:18.051525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:18.051525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:18.051525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:18.051525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:18.052475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:18.052475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:18.052475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:18.052475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:18.052475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:18.052475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:18.052475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:18.052475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:18.052475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:18.052475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:18.052475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:18.052475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:18.052475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:18.052475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:18.052475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:18.052475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:18.052475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:18.054403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:18.054403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:18.054403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:18.054403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:18.054645 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 235/472] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:18.055847 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b078aed-a67d-4e5f-aacb-232b35f3c1cc None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:18.056132 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b078aed-a67d-4e5f-aacb-232b35f3c1cc None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:18.056415 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b078aed-a67d-4e5f-aacb-232b35f3c1cc None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:18.058137 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 236/473] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:18.060341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b078aed-a67d-4e5f-aacb-232b35f3c1cc None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:18.085913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae92b88-ab49-4bbd-8360-165893ae31d7 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:18.117668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:18.126498 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4254c6c-d87e-4904-bcf6-4af8daf4a16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:18.127465 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4254c6c-d87e-4904-bcf6-4af8daf4a16e None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:18.127906 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4254c6c-d87e-4904-bcf6-4af8daf4a16e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:18.128402 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4254c6c-d87e-4904-bcf6-4af8daf4a16e None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:18.129350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4254c6c-d87e-4904-bcf6-4af8daf4a16e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:18.131040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4254c6c-d87e-4904-bcf6-4af8daf4a16e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:18.164366 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 238/474] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 17:21:17 2022] POST /identity/v3/users => generated 248 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:21:18.165935 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbd8d9-7a7b-49e0-bc43-ce20e4b875c8 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:18.166460 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbd8d9-7a7b-49e0-bc43-ce20e4b875c8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:18.167017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbd8d9-7a7b-49e0-bc43-ce20e4b875c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:18.195656 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e8dbd8d9-7a7b-49e0-bc43-ce20e4b875c8 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:18.196556 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b078aed-a67d-4e5f-aacb-232b35f3c1cc None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:18.198757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e8dbd8d9-7a7b-49e0-bc43-ce20e4b875c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:18.248457 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589b353-5951-4f3f-9f1a-d4d7e016d63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:18.249537 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589b353-5951-4f3f-9f1a-d4d7e016d63f None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:18.249962 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589b353-5951-4f3f-9f1a-d4d7e016d63f None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:18.250404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589b353-5951-4f3f-9f1a-d4d7e016d63f None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:18.252392 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6589b353-5951-4f3f-9f1a-d4d7e016d63f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:18.252892 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6589b353-5951-4f3f-9f1a-d4d7e016d63f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:18.254713 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6589b353-5951-4f3f-9f1a-d4d7e016d63f None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:18.270205 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:18.276168 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 237/475] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:18.281349 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589b353-5951-4f3f-9f1a-d4d7e016d63f None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:18.304702 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 239/476] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:18.343710 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dbd8d9-7a7b-49e0-bc43-ce20e4b875c8 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:18.407579 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:18.413327 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589b353-5951-4f3f-9f1a-d4d7e016d63f None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:18.451810 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b56c8-70db-4980-8412-6c6a704a88a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:18.452502 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b56c8-70db-4980-8412-6c6a704a88a0 None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:18.452793 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b56c8-70db-4980-8412-6c6a704a88a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:18.453083 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b56c8-70db-4980-8412-6c6a704a88a0 None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:18.453840 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b56c8-70db-4980-8412-6c6a704a88a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:18.455624 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b56c8-70db-4980-8412-6c6a704a88a0 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:18.479459 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 238/477] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 17:21:18 2022] POST /identity/v3/users => generated 249 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:21:18.526749 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dbd8d9-7a7b-49e0-bc43-ce20e4b875c8 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:18.571065 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699ce38-15a5-4889-8151-8a866da312d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:18.571760 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699ce38-15a5-4889-8151-8a866da312d3 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:18.572051 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699ce38-15a5-4889-8151-8a866da312d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:18.572334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699ce38-15a5-4889-8151-8a866da312d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:18.573117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699ce38-15a5-4889-8151-8a866da312d3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:18.574577 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699ce38-15a5-4889-8151-8a866da312d3 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:18.582677 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.sql.core [None req-3699ce38-15a5-4889-8151-8a866da312d3 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 17:21:18.582677 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 17:21:18.582677 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [parameters: {'id': '212c378bc44f46bdbebe1b25c4115afb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 17:21:18.582677 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80151) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 17:21:18.600581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-3699ce38-15a5-4889-8151-8a866da312d3 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 17:21:18.600581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:18.600581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:21:18.600581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:21:18.600581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:21:18.600581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:21:18.600581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:21:18.600581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:21:18.600581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:21:18.600581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:21:18.600581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:21:18.600581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:21:18.600581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:21:18.600581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application result.read() Jun 16 17:21:18.600581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:21:18.602159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:21:18.602159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:21:18.602159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:21:18.602159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:21:18.602159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:21:18.602159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:21:18.602159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:21:18.602159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 17:21:18.602159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:18.602159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 17:21:18.602159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:18.602159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:18.602159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 17:21:18.602159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 17:21:18.602159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 16 17:21:18.602159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return ref.to_dict() Jun 16 17:21:18.602159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:21:18.602159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:21:18.604377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 17:21:18.604377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application yield resource Jun 16 17:21:18.604377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:21:18.604377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:21:18.604377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 17:21:18.604377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 17:21:18.604377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 17:21:18.604377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 17:21:18.604377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 17:21:18.604377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise self.value Jun 16 17:21:18.604377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 17:21:18.604377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 17:21:18.604377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 17:21:18.604377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application session.commit() Jun 16 17:21:18.604377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 17:21:18.604377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 17:21:18.604377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 17:21:18.606175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 17:21:18.606175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 17:21:18.606175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self.session.flush() Jun 16 17:21:18.606175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 17:21:18.606175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 17:21:18.606175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 17:21:18.606175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 17:21:18.606175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 17:21:18.606175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application compat.raise_( Jun 16 17:21:18.606175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:21:18.606175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception Jun 16 17:21:18.606175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 17:21:18.606175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 17:21:18.606175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 17:21:18.606175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 17:21:18.606175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 17:21:18.606175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 17:21:18.608724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 17:21:18.608724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 17:21:18.608724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 17:21:18.608724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 17:21:18.608724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 17:21:18.608724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 17:21:18.608724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 17:21:18.608724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 17:21:18.608724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 17:21:18.608724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 17:21:18.608724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 17:21:18.608724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 17:21:18.608724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 17:21:18.608724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 17:21:18.608724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:21:18.608724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception Jun 16 17:21:18.612720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:21:18.612720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:21:18.612720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:21:18.612720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:21:18.612720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:21:18.612720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:21:18.612720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:21:18.612720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:21:18.612720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:21:18.612720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:21:18.612720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:21:18.612720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application result.read() Jun 16 17:21:18.612720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:21:18.612720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:21:18.612720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:21:18.612720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:21:18.617252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:21:18.617252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:21:18.617252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:21:18.617252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:21:18.617252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 17:21:18.617252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 17:21:18.617252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [parameters: {'id': '212c378bc44f46bdbebe1b25c4115afb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 17:21:18.617252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 17:21:18.617252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:18.617252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:21:18.617252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:18.617252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:18.617252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:18.617252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:18.617252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:18.617252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:18.621792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:18.621792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:18.621792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:18.621792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:18.621792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:18.621792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:18.621792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 16 17:21:18.621792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 16 17:21:18.621792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:18.621792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:18.621792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 16 17:21:18.621792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 16 17:21:18.621792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 17:21:18.621792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 17:21:18.621792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 17:21:18.621792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:18.624869 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 240/478] 10.222.0.40 () {60 vars in 1131 bytes} [Thu Jun 16 17:21:18 2022] POST /identity/v3/roles => generated 144 bytes in 259 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:21:18.625086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b56c8-70db-4980-8412-6c6a704a88a0 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:18.734364 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b56c8-70db-4980-8412-6c6a704a88a0 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:18.772419 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00cd3f2-49f6-446e-b98e-8c8c4bde18a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:18.773279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00cd3f2-49f6-446e-b98e-8c8c4bde18a1 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:18.773559 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00cd3f2-49f6-446e-b98e-8c8c4bde18a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:18.773838 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00cd3f2-49f6-446e-b98e-8c8c4bde18a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:18.785406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00cd3f2-49f6-446e-b98e-8c8c4bde18a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:18.793571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-b00cd3f2-49f6-446e-b98e-8c8c4bde18a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:21:18.793571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:18.793571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:18.793571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:18.793571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:18.793571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:18.793571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:18.793571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:18.793571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:18.793571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:18.793571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:18.793571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:18.793571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:18.793571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:18.793571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:18.793571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:18.796413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:18.796413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:18.796413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:18.796413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:18.796413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:18.796413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:18.796413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:18.796413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:18.796413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:18.796413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:18.796413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:18.796413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:18.796413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:18.796413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:18.796413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:18.796413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:18.798301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:18.798301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:18.798301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:18.798301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:18.798301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:18.798301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:18.798301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:18.798301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:18.798301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:18.798301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:18.798301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:21:18.798301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:18.798301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 239/479] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:21:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:18.802433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699ce38-15a5-4889-8151-8a866da312d3 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:18.854923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:18.926873 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699ce38-15a5-4889-8151-8a866da312d3 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:18.974758 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e8bc6-dd93-43a7-a148-62e405fd0082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:18.976211 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e8bc6-dd93-43a7-a148-62e405fd0082 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:18.976885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e8bc6-dd93-43a7-a148-62e405fd0082 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:18.977582 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e8bc6-dd93-43a7-a148-62e405fd0082 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:18.979012 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e8bc6-dd93-43a7-a148-62e405fd0082 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:18.979729 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e8bc6-dd93-43a7-a148-62e405fd0082 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:18.982214 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e8bc6-dd93-43a7-a148-62e405fd0082 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:19.005831 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 241/480] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:20.156469 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac1829-f618-425f-a948-52dbb193e510 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:20.157478 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac1829-f618-425f-a948-52dbb193e510 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:20.157974 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac1829-f618-425f-a948-52dbb193e510 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:20.159808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 240/481] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:20.170773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7299ead-8eb2-4163-b262-ef6766f4df9c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:20.171641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7299ead-8eb2-4163-b262-ef6766f4df9c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:20.172109 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7299ead-8eb2-4163-b262-ef6766f4df9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:20.194159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d7299ead-8eb2-4163-b262-ef6766f4df9c None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:20.195383 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d7299ead-8eb2-4163-b262-ef6766f4df9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:20.300422 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 242/482] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:20.331058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac1829-f618-425f-a948-52dbb193e510 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:20.358500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:20.481875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac1829-f618-425f-a948-52dbb193e510 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:20.522623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ef3b8-b7bd-48dc-8261-497e1862a333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:20.523439 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ef3b8-b7bd-48dc-8261-497e1862a333 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:20.523731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ef3b8-b7bd-48dc-8261-497e1862a333 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:20.524014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ef3b8-b7bd-48dc-8261-497e1862a333 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:20.534989 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ef3b8-b7bd-48dc-8261-497e1862a333 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:20.543029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-2e4ef3b8-b7bd-48dc-8261-497e1862a333 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:20.543029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:20.543029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:20.543029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:20.543029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:20.543029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:20.543029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:20.543029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:20.543029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:20.543029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:20.543029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:20.543029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:20.543029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:20.543029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:20.543029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:20.543029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:20.544014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:20.544014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:20.544014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:20.544014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:20.544014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:20.544014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:20.544014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:20.544014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:20.544014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:20.544014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:20.544014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:20.544014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:20.544014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:20.544014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:20.544014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:20.544014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:20.544866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:20.544866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:20.544866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:20.544866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:20.544866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:20.544866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:20.544866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:20.544866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:20.544866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:20.544866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:20.544866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:20.544866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:20.544866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 241/483] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:21:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:20.550369 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7299ead-8eb2-4163-b262-ef6766f4df9c None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:20.604715 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:20.708149 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7299ead-8eb2-4163-b262-ef6766f4df9c None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:20.758151 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199f4c6-9d5f-43d6-ab1d-0264aee5dbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:20.759505 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199f4c6-9d5f-43d6-ab1d-0264aee5dbf9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:20.760016 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199f4c6-9d5f-43d6-ab1d-0264aee5dbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:20.760509 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199f4c6-9d5f-43d6-ab1d-0264aee5dbf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:20.761500 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199f4c6-9d5f-43d6-ab1d-0264aee5dbf9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:20.762007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199f4c6-9d5f-43d6-ab1d-0264aee5dbf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:20.763582 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199f4c6-9d5f-43d6-ab1d-0264aee5dbf9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:20.776723 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 243/484] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 17:21:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:20.782999 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ef3b8-b7bd-48dc-8261-497e1862a333 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:20.880787 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:20.907422 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ef3b8-b7bd-48dc-8261-497e1862a333 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:20.945626 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825b0c4-1a06-4dc9-9622-f247dbd2f952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:20.946449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825b0c4-1a06-4dc9-9622-f247dbd2f952 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:20.946742 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825b0c4-1a06-4dc9-9622-f247dbd2f952 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:20.947027 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825b0c4-1a06-4dc9-9622-f247dbd2f952 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:20.961115 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0825b0c4-1a06-4dc9-9622-f247dbd2f952 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:20.962659 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0825b0c4-1a06-4dc9-9622-f247dbd2f952 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:20.981347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-0825b0c4-1a06-4dc9-9622-f247dbd2f952 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:20.981347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:20.981347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:20.981347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:20.981347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:20.981347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:20.981347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:20.981347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:20.981347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:20.981347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:20.981347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:20.981347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:20.981347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:20.981347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:20.981347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:20.982291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:20.982291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:20.982291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:20.982291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:20.982291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:20.982291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:20.982291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:20.982291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:20.982291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:20.982291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:20.982291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:20.982291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:20.982291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:20.982291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:20.982291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:20.982291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:20.982291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:20.983158 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:20.983158 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:20.983158 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:20.983158 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:20.983158 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 242/485] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:20.989300 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199f4c6-9d5f-43d6-ab1d-0264aee5dbf9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:21.038402 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5803ca0-095f-45e3-b3e2-f3f4c1bb47ec None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:21.038709 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5803ca0-095f-45e3-b3e2-f3f4c1bb47ec None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:21.038987 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5803ca0-095f-45e3-b3e2-f3f4c1bb47ec None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:21.040364 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 243/486] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:21.050919 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16846364-5dae-4b11-a786-5e941345ba7b None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:21.051219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16846364-5dae-4b11-a786-5e941345ba7b None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:21.051515 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16846364-5dae-4b11-a786-5e941345ba7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:21.086430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-16846364-5dae-4b11-a786-5e941345ba7b None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:21.087774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-16846364-5dae-4b11-a786-5e941345ba7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:21.116987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:21.123152 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199f4c6-9d5f-43d6-ab1d-0264aee5dbf9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:21.165052 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a6fa0-df95-4c2c-9310-03f9e1ebd342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:21.165756 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a6fa0-df95-4c2c-9310-03f9e1ebd342 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:21.166246 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a6fa0-df95-4c2c-9310-03f9e1ebd342 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:21.166678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a6fa0-df95-4c2c-9310-03f9e1ebd342 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:21.167562 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a6fa0-df95-4c2c-9310-03f9e1ebd342 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:21.168030 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a6fa0-df95-4c2c-9310-03f9e1ebd342 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:21.169594 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a6fa0-df95-4c2c-9310-03f9e1ebd342 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:21.170087 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 244/487] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 16 17:21:21.172258 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9c112-2204-4dba-bf63-f9d5bd2ca194 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:21.172735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9c112-2204-4dba-bf63-f9d5bd2ca194 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:21.173215 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9c112-2204-4dba-bf63-f9d5bd2ca194 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:21.174946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 245/488] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:21.179922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562793f-4b78-4569-aceb-6e7b272ef112 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:21.180378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562793f-4b78-4569-aceb-6e7b272ef112 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:21.180806 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562793f-4b78-4569-aceb-6e7b272ef112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:21.183122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 244/489] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:21.187976 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a6fa0-df95-4c2c-9310-03f9e1ebd342 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:21.218304 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-6562793f-4b78-4569-aceb-6e7b272ef112 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:21.219848 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-6562793f-4b78-4569-aceb-6e7b272ef112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:21.289810 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:21.315493 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 246/490] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:21:21.317100 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562793f-4b78-4569-aceb-6e7b272ef112 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:21.332012 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a6fa0-df95-4c2c-9310-03f9e1ebd342 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:21.383487 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbc664-7848-4fb5-86b7-0a9b2cb38c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:21.384913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbc664-7848-4fb5-86b7-0a9b2cb38c35 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:21.385396 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbc664-7848-4fb5-86b7-0a9b2cb38c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:21.385821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbc664-7848-4fb5-86b7-0a9b2cb38c35 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:21.394477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cbc664-7848-4fb5-86b7-0a9b2cb38c35 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:21.396371 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cbc664-7848-4fb5-86b7-0a9b2cb38c35 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:21.418062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-37cbc664-7848-4fb5-86b7-0a9b2cb38c35 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:21:21.418062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:21.418062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:21.418062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:21.418062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:21.418062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:21.418062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:21.418062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:21.418062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:21.418062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:21.418062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:21.418062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:21.418062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:21.418062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:21.418062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:21.418062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:21.419042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:21.419042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:21.419042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:21.419042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:21.419042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:21.419042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:21.419042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:21.419042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:21.419042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:21.419042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:21.419042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:21.419042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:21.419042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:21.419042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:21.419042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:21.419042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:21.419042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:21.419042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:21.419984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:21.419984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:21.419984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:21.419984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:21.419984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:21.419984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:21.419984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:21.419984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:21.419984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:21:21.419984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:21.427621 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 245/491] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jun 16 17:21:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:21:21.427621 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbc664-7848-4fb5-86b7-0a9b2cb38c35 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:21.447672 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:21.448656 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562793f-4b78-4569-aceb-6e7b272ef112 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:21.485896 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdda9a-272e-4fee-9c48-f751efdd288b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:21.486983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdda9a-272e-4fee-9c48-f751efdd288b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:21.487421 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdda9a-272e-4fee-9c48-f751efdd288b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:21.487875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdda9a-272e-4fee-9c48-f751efdd288b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:21.502674 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbdda9a-272e-4fee-9c48-f751efdd288b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:21.516206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-ccbdda9a-272e-4fee-9c48-f751efdd288b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:21.516206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:21.516206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:21.516206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:21.516206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:21.516206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:21.516206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:21.516206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:21.516206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:21.516206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:21.516206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:21.516206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:21.516206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:21.516206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:21.516206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:21.516206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:21.518032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:21.518032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:21.518032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:21.518032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:21.518032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:21.518032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:21.518032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:21.518032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:21.518032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:21.518032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:21.518032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:21.518032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:21.518032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:21.518032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:21.518032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:21.518032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:21.519133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:21.519133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:21.519133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:21.519133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:21.519133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:21.519133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:21.519133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:21.519133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:21.519133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:21.519133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:21.519133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:21.519133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:21.526970 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 247/492] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:21:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:21.526970 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdda9a-272e-4fee-9c48-f751efdd288b None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:21.556422 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbc664-7848-4fb5-86b7-0a9b2cb38c35 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:21.582582 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:21.604095 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a05f0-9f2a-4918-b872-4b2378638ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:21.604901 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a05f0-9f2a-4918-b872-4b2378638ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:21.605217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a05f0-9f2a-4918-b872-4b2378638ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:21.605507 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a05f0-9f2a-4918-b872-4b2378638ed1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:21.617853 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a05f0-9f2a-4918-b872-4b2378638ed1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:21.626292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-d31a05f0-9f2a-4918-b872-4b2378638ed1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:21.626292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:21.626292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:21.626292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:21.626292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:21.626292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:21.626292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:21.626292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:21.626292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:21.626292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:21.626292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:21.626292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:21.626292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:21.626292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:21.626292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:21.626292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:21.627446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:21.627446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:21.627446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:21.627446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:21.627446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:21.627446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:21.627446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:21.627446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:21.627446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:21.627446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:21.627446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:21.627446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:21.627446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:21.627446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:21.627446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:21.627446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:21.628894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:21.628894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:21.628894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:21.628894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:21.628894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:21.628894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:21.628894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:21.628894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:21.628894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:21.628894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:21.628894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:21.628894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:21.632287 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 246/493] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:21:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:21.633010 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a05f0-9f2a-4918-b872-4b2378638ed1 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:21.643796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdda9a-272e-4fee-9c48-f751efdd288b None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:21.684876 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c003b5-ddfb-48ce-9e49-92e38d3081c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:21.685610 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c003b5-ddfb-48ce-9e49-92e38d3081c6 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:21.685894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c003b5-ddfb-48ce-9e49-92e38d3081c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:21.686178 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c003b5-ddfb-48ce-9e49-92e38d3081c6 None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:21.687109 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c003b5-ddfb-48ce-9e49-92e38d3081c6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:21.687483 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c003b5-ddfb-48ce-9e49-92e38d3081c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:21.689010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c003b5-ddfb-48ce-9e49-92e38d3081c6 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:21.716363 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:21.729298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 248/494] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jun 16 17:21:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:21.730841 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ea5e9-5c7e-4e95-8c12-883c7304a744 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:21.731210 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ea5e9-5c7e-4e95-8c12-883c7304a744 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:21.731573 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ea5e9-5c7e-4e95-8c12-883c7304a744 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:21.733334 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 249/495] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:21.738115 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ea5e9-5c7e-4e95-8c12-883c7304a744 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:21.744080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a05f0-9f2a-4918-b872-4b2378638ed1 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:21.786275 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52cffcb-ab76-4e37-baa3-3dd2657f270e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:21.786955 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52cffcb-ab76-4e37-baa3-3dd2657f270e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:21.787240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52cffcb-ab76-4e37-baa3-3dd2657f270e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:21.787516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52cffcb-ab76-4e37-baa3-3dd2657f270e None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:21.788255 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52cffcb-ab76-4e37-baa3-3dd2657f270e None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:21.788571 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52cffcb-ab76-4e37-baa3-3dd2657f270e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:21.789973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52cffcb-ab76-4e37-baa3-3dd2657f270e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:21.799439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 247/496] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 17:21:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:21.800521 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52cffcb-ab76-4e37-baa3-3dd2657f270e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:21.862736 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ea5e9-5c7e-4e95-8c12-883c7304a744 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:21.866472 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:21.902801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d10364-0863-49cf-96d2-0e5bc668d9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:21.903471 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d10364-0863-49cf-96d2-0e5bc668d9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:21.903760 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d10364-0863-49cf-96d2-0e5bc668d9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:21.904042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d10364-0863-49cf-96d2-0e5bc668d9c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:21.904793 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d10364-0863-49cf-96d2-0e5bc668d9c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:21.905103 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d10364-0863-49cf-96d2-0e5bc668d9c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:21.906494 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d10364-0863-49cf-96d2-0e5bc668d9c5 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:21.911839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52cffcb-ab76-4e37-baa3-3dd2657f270e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:21.914983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 250/497] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 17:21:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:21.916177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206bdd7-f21f-40f8-86b1-6a6880db5637 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:21.916468 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206bdd7-f21f-40f8-86b1-6a6880db5637 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:21.916752 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206bdd7-f21f-40f8-86b1-6a6880db5637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:21.932628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-d206bdd7-f21f-40f8-86b1-6a6880db5637 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:21.933832 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-d206bdd7-f21f-40f8-86b1-6a6880db5637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:21.945667 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96899c4-60ab-4d8f-8e6f-546e0f95108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:21.946327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96899c4-60ab-4d8f-8e6f-546e0f95108c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:21.946615 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96899c4-60ab-4d8f-8e6f-546e0f95108c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:21.946896 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96899c4-60ab-4d8f-8e6f-546e0f95108c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:21.956274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:22.002423 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 251/498] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:22.003252 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206bdd7-f21f-40f8-86b1-6a6880db5637 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:22.028348 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96899c4-60ab-4d8f-8e6f-546e0f95108c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:22.029832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96899c4-60ab-4d8f-8e6f-546e0f95108c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:22.098674 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:22.105539 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206bdd7-f21f-40f8-86b1-6a6880db5637 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:22.130727 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 248/499] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:21:22.132189 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155be24-0593-4233-aafd-78cb1787105d None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:22.132488 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155be24-0593-4233-aafd-78cb1787105d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:22.132774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155be24-0593-4233-aafd-78cb1787105d None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:22.134369 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 249/500] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:22.135062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155be24-0593-4233-aafd-78cb1787105d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:22.145458 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6e7db-5135-4717-bd75-e1c1e740bb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:22.146265 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6e7db-5135-4717-bd75-e1c1e740bb48 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:22.146560 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6e7db-5135-4717-bd75-e1c1e740bb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:22.146843 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6e7db-5135-4717-bd75-e1c1e740bb48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:22.156948 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6e7db-5135-4717-bd75-e1c1e740bb48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:22.159156 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6e7db-5135-4717-bd75-e1c1e740bb48 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:22.177534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-dbb6e7db-5135-4717-bd75-e1c1e740bb48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:22.177534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:22.177534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:22.177534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:22.177534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:22.177534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:22.177534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:22.177534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:22.177534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:22.177534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:22.177534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:22.177534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:22.177534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:22.177534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:22.177534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:22.179086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:22.179086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:22.179086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:22.179086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:22.179086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:22.179086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:22.179086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:22.179086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:22.179086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:22.179086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:22.179086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:22.179086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:22.179086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:22.179086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:22.179086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:22.179086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:22.179086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:22.179993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:22.179993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:22.179993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:22.179993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:22.180208 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:22.183285 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 252/501] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:22.183995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6e7db-5135-4717-bd75-e1c1e740bb48 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:22.242589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155be24-0593-4233-aafd-78cb1787105d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:22.277961 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90d03c-c9d4-43a0-93e3-f351616b6212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:22.278757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90d03c-c9d4-43a0-93e3-f351616b6212 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:22.279041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90d03c-c9d4-43a0-93e3-f351616b6212 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:22.279328 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90d03c-c9d4-43a0-93e3-f351616b6212 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:22.286463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6e7db-5135-4717-bd75-e1c1e740bb48 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:22.289844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90d03c-c9d4-43a0-93e3-f351616b6212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:22.291287 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90d03c-c9d4-43a0-93e3-f351616b6212 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:22.308391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-0d90d03c-c9d4-43a0-93e3-f351616b6212 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:22.308391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:22.308391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:22.308391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:22.308391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:22.308391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:22.308391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:22.308391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:22.308391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:22.308391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:22.308391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:22.308391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:22.308391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:22.308391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:22.308391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:22.309334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:22.309334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:22.309334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:22.309334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:22.309334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:22.309334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:22.309334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:22.309334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:22.309334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:22.309334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:22.309334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:22.309334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:22.309334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:22.309334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:22.309334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:22.309334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:22.309334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:22.310228 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:22.310228 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:22.310228 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:22.310228 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:22.310228 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 250/502] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:22.310496 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90d03c-c9d4-43a0-93e3-f351616b6212 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:22.324034 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c21b6-035e-45fa-9696-9d336b66fa97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:22.324830 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c21b6-035e-45fa-9696-9d336b66fa97 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:22.325122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c21b6-035e-45fa-9696-9d336b66fa97 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:22.325430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c21b6-035e-45fa-9696-9d336b66fa97 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:22.336016 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c21b6-035e-45fa-9696-9d336b66fa97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:22.343767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-4f0c21b6-035e-45fa-9696-9d336b66fa97 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:22.343767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:22.343767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:22.343767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:22.343767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:22.343767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:22.343767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:22.343767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:22.343767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:22.343767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:22.343767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:22.343767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:22.343767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:22.343767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:22.343767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:22.343767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:22.345057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:22.345057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:22.345057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:22.345057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:22.345057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:22.345057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:22.345057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:22.345057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:22.345057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:22.345057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:22.345057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:22.345057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:22.345057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:22.345057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:22.345057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:22.345057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:22.346177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:22.346177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:22.346177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:22.346177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:22.346177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:22.346177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:22.346177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:22.346177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:22.346177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:22.346177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:22.346177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:22.346177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:22.346177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 253/503] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:21:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:22.347489 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052b0c8-83dc-45ee-bf38-63b1fc1d67b6 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:22.347784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052b0c8-83dc-45ee-bf38-63b1fc1d67b6 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:22.348066 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052b0c8-83dc-45ee-bf38-63b1fc1d67b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:22.366065 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-e052b0c8-83dc-45ee-bf38-63b1fc1d67b6 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:22.367306 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-e052b0c8-83dc-45ee-bf38-63b1fc1d67b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:22.369341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:22.403771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:22.413835 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90d03c-c9d4-43a0-93e3-f351616b6212 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:22.439967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 254/504] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:22.441067 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052b0c8-83dc-45ee-bf38-63b1fc1d67b6 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:22.450651 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35943111-4538-4f9f-a348-2626cc95d3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:22.451441 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35943111-4538-4f9f-a348-2626cc95d3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:22.451727 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35943111-4538-4f9f-a348-2626cc95d3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:22.452006 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35943111-4538-4f9f-a348-2626cc95d3e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:22.462838 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35943111-4538-4f9f-a348-2626cc95d3e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:22.464688 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35943111-4538-4f9f-a348-2626cc95d3e1 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:22.482131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-35943111-4538-4f9f-a348-2626cc95d3e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:22.482131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:22.482131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:22.482131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:22.482131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:22.482131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:22.482131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:22.482131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:22.482131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:22.482131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:22.482131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:22.482131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:22.482131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:22.482131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:22.482131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:22.483063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:22.483063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:22.483063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:22.483063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:22.483063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:22.483063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:22.483063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:22.483063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:22.483063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:22.483063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:22.483063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:22.483063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:22.483063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:22.483063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:22.483063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:22.483063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:22.483063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:22.483961 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:22.483961 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:22.483961 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:22.483961 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:22.483961 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 251/505] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:22.484353 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35943111-4538-4f9f-a348-2626cc95d3e1 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:22.543545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052b0c8-83dc-45ee-bf38-63b1fc1d67b6 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:22.577765 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436df02-f9eb-44b8-af4a-19dd85773870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:22.578431 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436df02-f9eb-44b8-af4a-19dd85773870 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:22.578720 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436df02-f9eb-44b8-af4a-19dd85773870 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:22.579147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436df02-f9eb-44b8-af4a-19dd85773870 None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:22.579927 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436df02-f9eb-44b8-af4a-19dd85773870 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:22.580242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436df02-f9eb-44b8-af4a-19dd85773870 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:22.581692 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436df02-f9eb-44b8-af4a-19dd85773870 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:22.586815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35943111-4538-4f9f-a348-2626cc95d3e1 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:22.588522 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:22.594447 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 255/506] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:22.595761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436df02-f9eb-44b8-af4a-19dd85773870 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:22.625412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dace56-8397-4b00-b090-b7e8115ac36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:22.626213 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dace56-8397-4b00-b090-b7e8115ac36c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:22.626505 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dace56-8397-4b00-b090-b7e8115ac36c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:22.626790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dace56-8397-4b00-b090-b7e8115ac36c None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:22.627520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dace56-8397-4b00-b090-b7e8115ac36c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:22.627834 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dace56-8397-4b00-b090-b7e8115ac36c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:22.629253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dace56-8397-4b00-b090-b7e8115ac36c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:22.640745 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 252/507] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:22.641449 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dace56-8397-4b00-b090-b7e8115ac36c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:22.644250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:22.702023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436df02-f9eb-44b8-af4a-19dd85773870 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:22.735920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17e22f-addd-48ed-893b-41138a53bb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:22.736553 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17e22f-addd-48ed-893b-41138a53bb85 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:22.736846 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17e22f-addd-48ed-893b-41138a53bb85 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:22.737128 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17e22f-addd-48ed-893b-41138a53bb85 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:22.737916 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17e22f-addd-48ed-893b-41138a53bb85 None admin] RBAC: Adding query filter params (name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:22.738231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17e22f-addd-48ed-893b-41138a53bb85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:22.739605 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17e22f-addd-48ed-893b-41138a53bb85 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:22.743252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dace56-8397-4b00-b090-b7e8115ac36c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:22.748245 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 256/508] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 17:21:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:22.749039 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17e22f-addd-48ed-893b-41138a53bb85 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:22.777047 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d008d39-f14c-4530-9d91-9df048695d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:22.777880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d008d39-f14c-4530-9d91-9df048695d14 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:22.778174 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d008d39-f14c-4530-9d91-9df048695d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:22.778462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d008d39-f14c-4530-9d91-9df048695d14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:22.789343 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d008d39-f14c-4530-9d91-9df048695d14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:22.790802 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d008d39-f14c-4530-9d91-9df048695d14 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:22.808105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-1d008d39-f14c-4530-9d91-9df048695d14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:22.808105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:22.808105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:22.808105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:22.808105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:22.808105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:22.808105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:22.808105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:22.808105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:22.808105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:22.808105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:22.808105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:22.808105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:22.808105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:22.808105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:22.809036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:22.809036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:22.809036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:22.809036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:22.809036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:22.809036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:22.809036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:22.809036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:22.809036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:22.809036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:22.809036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:22.809036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:22.809036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:22.809036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:22.809036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:22.809036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:22.809036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:22.810646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:22.810646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:22.810646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:22.810646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:22.810646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 253/509] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:22.810646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d008d39-f14c-4530-9d91-9df048695d14 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:22.811436 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:22.853362 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17e22f-addd-48ed-893b-41138a53bb85 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:22.866609 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:22.887219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03593f-b665-424b-b340-8a2a676a53d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:22.887877 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03593f-b665-424b-b340-8a2a676a53d9 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:22.888163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03593f-b665-424b-b340-8a2a676a53d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:22.888442 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03593f-b665-424b-b340-8a2a676a53d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:22.889188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03593f-b665-424b-b340-8a2a676a53d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:22.889507 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03593f-b665-424b-b340-8a2a676a53d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:22.890875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03593f-b665-424b-b340-8a2a676a53d9 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:22.902478 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 257/510] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:22.903270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03593f-b665-424b-b340-8a2a676a53d9 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:22.912802 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d008d39-f14c-4530-9d91-9df048695d14 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:22.946493 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75905708-9fae-4357-9e8e-1b326ee18768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:22.947297 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75905708-9fae-4357-9e8e-1b326ee18768 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:22.947587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75905708-9fae-4357-9e8e-1b326ee18768 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:22.947869 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75905708-9fae-4357-9e8e-1b326ee18768 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:22.956516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75905708-9fae-4357-9e8e-1b326ee18768 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:22.957958 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75905708-9fae-4357-9e8e-1b326ee18768 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:22.972913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-75905708-9fae-4357-9e8e-1b326ee18768 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:21:22.972913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:22.972913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:22.972913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:22.972913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:22.972913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:22.972913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:22.972913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:22.972913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:22.972913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:22.972913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:22.972913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:22.972913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:22.972913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:22.972913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:22.972913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:22.973945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:22.973945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:22.973945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:22.973945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:22.973945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:22.973945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:22.973945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:22.973945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:22.973945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:22.973945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:22.973945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:22.973945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:22.973945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:22.973945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:22.973945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:22.973945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:22.973945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:22.973945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:22.974885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:22.974885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:22.974885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:22.974885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:22.974885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:22.974885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:22.974885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:22.974885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:22.974885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:21:22.974885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:22.974885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 254/511] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:22.975479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75905708-9fae-4357-9e8e-1b326ee18768 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:23.004338 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03593f-b665-424b-b340-8a2a676a53d9 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:23.024731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:23.039527 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadb3d2-d88c-444e-8f2c-511fef6810cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:23.040324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadb3d2-d88c-444e-8f2c-511fef6810cd None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:23.040613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadb3d2-d88c-444e-8f2c-511fef6810cd None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:23.040891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadb3d2-d88c-444e-8f2c-511fef6810cd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:23.048094 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadb3d2-d88c-444e-8f2c-511fef6810cd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:23.049595 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadb3d2-d88c-444e-8f2c-511fef6810cd None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:23.065336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-afadb3d2-d88c-444e-8f2c-511fef6810cd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:21:23.065336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:23.065336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:23.065336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:23.065336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:23.065336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:23.065336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:23.065336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:23.065336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:23.065336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:23.065336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:23.065336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:23.065336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:23.065336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:23.065336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:23.065336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:23.066497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:23.066497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:23.066497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:23.066497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:23.066497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:23.066497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:23.066497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:23.066497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:23.066497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:23.066497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:23.066497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:23.066497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:23.066497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:23.066497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:23.066497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:23.066497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:23.066497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:23.066497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:23.067966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:23.067966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:23.067966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:23.067966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:23.067966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:23.067966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:23.067966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:23.067966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:23.067966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:21:23.067966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:23.067966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 258/512] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:23.067966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadb3d2-d88c-444e-8f2c-511fef6810cd None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:23.079377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75905708-9fae-4357-9e8e-1b326ee18768 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:23.092955 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:23.113893 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98520646-5854-49f4-98a1-7650b4f38575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:23.114688 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98520646-5854-49f4-98a1-7650b4f38575 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:23.114972 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98520646-5854-49f4-98a1-7650b4f38575 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:23.115246 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98520646-5854-49f4-98a1-7650b4f38575 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:23.126040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98520646-5854-49f4-98a1-7650b4f38575 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:23.127752 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98520646-5854-49f4-98a1-7650b4f38575 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:23.145281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-98520646-5854-49f4-98a1-7650b4f38575 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:23.145281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:23.145281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:23.145281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:23.145281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:23.145281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:23.145281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:23.145281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:23.145281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:23.145281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:23.145281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:23.145281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:23.145281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:23.145281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:23.145281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:23.146355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:23.146355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:23.146355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:23.146355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:23.146355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:23.146355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:23.146355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:23.146355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:23.146355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:23.146355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:23.146355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:23.146355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:23.146355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:23.146355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:23.146355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:23.146355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:23.146355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:23.147463 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:23.147463 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:23.147463 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:23.147463 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:23.147463 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 255/513] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:23.147870 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98520646-5854-49f4-98a1-7650b4f38575 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:23.173411 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadb3d2-d88c-444e-8f2c-511fef6810cd None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:23.207910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0347547-8de6-4058-8740-64707ef542b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:23.208539 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0347547-8de6-4058-8740-64707ef542b6 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:23.208784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0347547-8de6-4058-8740-64707ef542b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:23.209068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0347547-8de6-4058-8740-64707ef542b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:23.209846 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0347547-8de6-4058-8740-64707ef542b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:23.210166 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0347547-8de6-4058-8740-64707ef542b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:23.211550 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0347547-8de6-4058-8740-64707ef542b6 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:23.223211 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 259/514] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:23.224012 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0347547-8de6-4058-8740-64707ef542b6 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:23.258672 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98520646-5854-49f4-98a1-7650b4f38575 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:23.267796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:23.293233 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd2debf-4b79-48ca-a289-5b619670c36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:23.294042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2debf-4b79-48ca-a289-5b619670c36f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:23.294335 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2debf-4b79-48ca-a289-5b619670c36f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:23.294619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2debf-4b79-48ca-a289-5b619670c36f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:23.301876 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd2debf-4b79-48ca-a289-5b619670c36f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:23.303678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd2debf-4b79-48ca-a289-5b619670c36f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:23.319086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-8cd2debf-4b79-48ca-a289-5b619670c36f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:23.319086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:23.319086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:23.319086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:23.319086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:23.319086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:23.319086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:23.319086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:23.319086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:23.319086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:23.319086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:23.319086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:23.319086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:23.319086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:23.319086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:23.320001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:23.320001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:23.320001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:23.320001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:23.320001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:23.320001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:23.320001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:23.320001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:23.320001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:23.320001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:23.320001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:23.320001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:23.320001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:23.320001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:23.320001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:23.320001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:23.320001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:23.320979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:23.320979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:23.320979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:23.320979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:23.320979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:23.320979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:23.320979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:23.320979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:23.320979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 256/515] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:23.321516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd2debf-4b79-48ca-a289-5b619670c36f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:23.330811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0347547-8de6-4058-8740-64707ef542b6 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:23.348612 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:23.365782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3725c-e5a4-4ea2-b999-d55930bd0b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:23.366457 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3725c-e5a4-4ea2-b999-d55930bd0b5f None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:23.366748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3725c-e5a4-4ea2-b999-d55930bd0b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:23.367036 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3725c-e5a4-4ea2-b999-d55930bd0b5f None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:23.367973 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3725c-e5a4-4ea2-b999-d55930bd0b5f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:23.368296 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3725c-e5a4-4ea2-b999-d55930bd0b5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:23.369777 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3725c-e5a4-4ea2-b999-d55930bd0b5f None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:23.388143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 260/516] 10.222.0.40 () {56 vars in 1137 bytes} [Thu Jun 16 17:21:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:23.388960 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3725c-e5a4-4ea2-b999-d55930bd0b5f None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:23.424384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd2debf-4b79-48ca-a289-5b619670c36f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:23.458272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad37a9-021d-4071-9b30-b22632870b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:23.458873 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad37a9-021d-4071-9b30-b22632870b3d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:23.459163 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad37a9-021d-4071-9b30-b22632870b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:23.459443 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad37a9-021d-4071-9b30-b22632870b3d None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:23.460273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ad37a9-021d-4071-9b30-b22632870b3d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:23.460584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ad37a9-021d-4071-9b30-b22632870b3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:23.462041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ad37a9-021d-4071-9b30-b22632870b3d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:23.486166 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:23.490371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3725c-e5a4-4ea2-b999-d55930bd0b5f None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:23.497931 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 257/517] 10.222.0.40 () {56 vars in 1137 bytes} [Thu Jun 16 17:21:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:23.498769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad37a9-021d-4071-9b30-b22632870b3d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:23.532499 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813bb283-1186-482c-9547-9fa8a9670bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:23.533194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813bb283-1186-482c-9547-9fa8a9670bbb None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:23.533490 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813bb283-1186-482c-9547-9fa8a9670bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:23.533769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813bb283-1186-482c-9547-9fa8a9670bbb None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:23.534494 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813bb283-1186-482c-9547-9fa8a9670bbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:23.534827 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813bb283-1186-482c-9547-9fa8a9670bbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:23.536171 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813bb283-1186-482c-9547-9fa8a9670bbb None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:23.547509 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 261/518] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:23.548334 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813bb283-1186-482c-9547-9fa8a9670bbb None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:23.589520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:23.605859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad37a9-021d-4071-9b30-b22632870b3d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:23.639635 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef3826-8520-48a4-8221-13721e3f2c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:23.640303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef3826-8520-48a4-8221-13721e3f2c0a None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:23.640590 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef3826-8520-48a4-8221-13721e3f2c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:23.640869 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef3826-8520-48a4-8221-13721e3f2c0a None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:23.642202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef3826-8520-48a4-8221-13721e3f2c0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:23.643835 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef3826-8520-48a4-8221-13721e3f2c0a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:23.648173 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813bb283-1186-482c-9547-9fa8a9670bbb None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:23.663970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 258/519] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 17:21:23 2022] POST /identity/v3/users => generated 247 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:21:23.665075 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef3826-8520-48a4-8221-13721e3f2c0a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:23.695035 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d074d-9a78-400e-adb9-6573aa44a62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:23.695734 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d074d-9a78-400e-adb9-6573aa44a62d None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:23.696027 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d074d-9a78-400e-adb9-6573aa44a62d None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:23.696310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d074d-9a78-400e-adb9-6573aa44a62d None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:23.697132 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d074d-9a78-400e-adb9-6573aa44a62d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:23.697492 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d074d-9a78-400e-adb9-6573aa44a62d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:23.698985 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d074d-9a78-400e-adb9-6573aa44a62d None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:23.718734 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 262/520] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:23.719578 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d074d-9a78-400e-adb9-6573aa44a62d None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:23.741639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:23.794576 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef3826-8520-48a4-8221-13721e3f2c0a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:23.833748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d074d-9a78-400e-adb9-6573aa44a62d None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:23.837960 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00d0eb-bc3b-4057-93bc-414966805358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:23.838796 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00d0eb-bc3b-4057-93bc-414966805358 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:23.839678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00d0eb-bc3b-4057-93bc-414966805358 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:23.839678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00d0eb-bc3b-4057-93bc-414966805358 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:23.851047 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:23.877379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5b4b4-de9c-4657-9c33-c559644931d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:23.878076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5b4b4-de9c-4657-9c33-c559644931d3 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:23.878367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5b4b4-de9c-4657-9c33-c559644931d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:23.878647 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5b4b4-de9c-4657-9c33-c559644931d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:23.916975 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00d0eb-bc3b-4057-93bc-414966805358 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:23.918492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00d0eb-bc3b-4057-93bc-414966805358 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:23.955915 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5b4b4-de9c-4657-9c33-c559644931d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:23.957516 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5b4b4-de9c-4657-9c33-c559644931d3 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:23.976994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:24.018707 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 259/521] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:21:24.023676 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00d0eb-bc3b-4057-93bc-414966805358 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:24.066260 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:24.078514 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 263/522] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:21:24.078514 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5b4b4-de9c-4657-9c33-c559644931d3 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:24.136683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00d0eb-bc3b-4057-93bc-414966805358 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:24.172334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67cab8-0080-4617-a122-16cb0bc0e9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:24.173931 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67cab8-0080-4617-a122-16cb0bc0e9b4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:24.174221 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67cab8-0080-4617-a122-16cb0bc0e9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:24.174503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67cab8-0080-4617-a122-16cb0bc0e9b4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:24.182192 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67cab8-0080-4617-a122-16cb0bc0e9b4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:24.183414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67cab8-0080-4617-a122-16cb0bc0e9b4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:24.187893 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5b4b4-de9c-4657-9c33-c559644931d3 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:24.197939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-0c67cab8-0080-4617-a122-16cb0bc0e9b4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:21:24.197939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:24.197939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:24.197939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:24.197939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:24.197939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:24.197939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:24.197939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:24.197939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:24.197939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:24.197939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:24.197939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:24.197939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:24.197939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:24.197939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:24.197939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:24.198997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:24.198997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:24.198997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:24.198997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:24.198997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:24.198997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:24.198997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:24.198997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:24.198997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:24.198997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:24.198997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:24.198997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:24.198997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:24.198997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:24.198997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:24.198997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:24.198997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:24.198997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:24.200110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:24.200110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:24.200110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:24.200110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:24.200110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:24.200110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:24.200110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:24.200110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:24.200110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:21:24.200110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:24.203518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 260/523] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:21:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:24.204448 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67cab8-0080-4617-a122-16cb0bc0e9b4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:24.210636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:24.233479 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93781b0-f222-4271-a0c1-8f3f2a9f4842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:24.234362 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93781b0-f222-4271-a0c1-8f3f2a9f4842 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:24.234790 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93781b0-f222-4271-a0c1-8f3f2a9f4842 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:24.235846 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93781b0-f222-4271-a0c1-8f3f2a9f4842 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:24.237437 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93781b0-f222-4271-a0c1-8f3f2a9f4842 None admin] RBAC: Adding query filter params (role.id=0e34a6f9b89a4e29951fb189abacb34a, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=689a31e26c6b454a91c3fd79678b4d07) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:24.238004 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93781b0-f222-4271-a0c1-8f3f2a9f4842 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:24.240070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93781b0-f222-4271-a0c1-8f3f2a9f4842 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:24.247866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 264/524] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:21:24 2022] GET /identity/v3/role_assignments?user.id=689a31e26c6b454a91c3fd79678b4d07&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=0e34a6f9b89a4e29951fb189abacb34a => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:24.250066 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93781b0-f222-4271-a0c1-8f3f2a9f4842 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:24.323372 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:24.326436 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67cab8-0080-4617-a122-16cb0bc0e9b4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:24.366498 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9fb852-4b5e-417d-a359-75db39c09ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:24.367439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fb852-4b5e-417d-a359-75db39c09ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:24.367766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fb852-4b5e-417d-a359-75db39c09ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:24.368045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fb852-4b5e-417d-a359-75db39c09ff7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:24.379561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93781b0-f222-4271-a0c1-8f3f2a9f4842 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:24.379764 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9fb852-4b5e-417d-a359-75db39c09ff7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:24.381340 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9fb852-4b5e-417d-a359-75db39c09ff7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:24.401354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-4f9fb852-4b5e-417d-a359-75db39c09ff7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:24.401354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:24.401354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:24.401354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:24.401354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:24.401354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:24.401354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:24.401354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:24.401354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:24.401354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:24.401354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:24.401354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:24.401354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:24.401354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:24.401354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:24.402323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:24.402323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:24.402323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:24.402323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:24.402323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:24.402323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:24.402323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:24.402323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:24.402323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:24.402323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:24.402323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:24.402323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:24.402323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:24.402323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:24.402323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:24.402323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:24.402323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:24.403303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:24.403303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:24.403303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:24.403303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:24.408572 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 261/525] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:24.409552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9fb852-4b5e-417d-a359-75db39c09ff7 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:24.430748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6954b-e1c8-444b-a7a1-d3100e275f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:24.431571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6954b-e1c8-444b-a7a1-d3100e275f40 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:24.431866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6954b-e1c8-444b-a7a1-d3100e275f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:24.432152 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6954b-e1c8-444b-a7a1-d3100e275f40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:24.445430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c6954b-e1c8-444b-a7a1-d3100e275f40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:24.446962 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c6954b-e1c8-444b-a7a1-d3100e275f40 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:24.470594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-93c6954b-e1c8-444b-a7a1-d3100e275f40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:24.470594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:24.470594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:24.470594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:24.470594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:24.470594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:24.470594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:24.470594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:24.470594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:24.470594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:24.470594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:24.470594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:24.470594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:24.470594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:24.470594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:24.471943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:24.471943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:24.471943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:24.471943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:24.471943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:24.471943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:24.471943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:24.471943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:24.471943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:24.471943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:24.471943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:24.471943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:24.471943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:24.471943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:24.471943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:24.471943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:24.471943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:24.472817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:24.472817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:24.472817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:24.472817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:24.476750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 265/526] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:24.477692 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6954b-e1c8-444b-a7a1-d3100e275f40 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:24.515611 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:24.546254 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9fb852-4b5e-417d-a359-75db39c09ff7 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:24.581421 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58409bb8-cc4f-48aa-bde6-211c60d1c0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:24.582111 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58409bb8-cc4f-48aa-bde6-211c60d1c0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:24.582403 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58409bb8-cc4f-48aa-bde6-211c60d1c0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:24.582684 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58409bb8-cc4f-48aa-bde6-211c60d1c0e3 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:24.583541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58409bb8-cc4f-48aa-bde6-211c60d1c0e3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:24.583856 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58409bb8-cc4f-48aa-bde6-211c60d1c0e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:24.585349 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58409bb8-cc4f-48aa-bde6-211c60d1c0e3 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:24.600082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6954b-e1c8-444b-a7a1-d3100e275f40 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:24.608905 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 262/527] 10.222.0.40 () {56 vars in 1139 bytes} [Thu Jun 16 17:21:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:24.609862 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58409bb8-cc4f-48aa-bde6-211c60d1c0e3 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:24.622300 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:24.644466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933dd49-8b4d-4ecf-80dd-b5f8cc2bd8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:24.645671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933dd49-8b4d-4ecf-80dd-b5f8cc2bd8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:24.646170 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933dd49-8b4d-4ecf-80dd-b5f8cc2bd8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:24.646702 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933dd49-8b4d-4ecf-80dd-b5f8cc2bd8a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:24.647700 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933dd49-8b4d-4ecf-80dd-b5f8cc2bd8a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:24.648239 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933dd49-8b4d-4ecf-80dd-b5f8cc2bd8a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:24.649986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933dd49-8b4d-4ecf-80dd-b5f8cc2bd8a4 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:24.665520 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 266/528] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:24.666914 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933dd49-8b4d-4ecf-80dd-b5f8cc2bd8a4 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:24.723974 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58409bb8-cc4f-48aa-bde6-211c60d1c0e3 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:24.772501 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93444b69-fdac-4e1b-8e7d-2ee43ecd2362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:24.773565 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93444b69-fdac-4e1b-8e7d-2ee43ecd2362 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:24.774508 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93444b69-fdac-4e1b-8e7d-2ee43ecd2362 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:24.774987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93444b69-fdac-4e1b-8e7d-2ee43ecd2362 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:24.776180 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93444b69-fdac-4e1b-8e7d-2ee43ecd2362 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:24.776943 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93444b69-fdac-4e1b-8e7d-2ee43ecd2362 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:24.779276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93444b69-fdac-4e1b-8e7d-2ee43ecd2362 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:24.780008 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:24.788274 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933dd49-8b4d-4ecf-80dd-b5f8cc2bd8a4 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:24.798974 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 263/529] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:24.800313 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93444b69-fdac-4e1b-8e7d-2ee43ecd2362 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:24.836118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9eff8-9409-4f1b-9343-f673788e1d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:24.837042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9eff8-9409-4f1b-9343-f673788e1d66 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:24.837374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9eff8-9409-4f1b-9343-f673788e1d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:24.837687 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9eff8-9409-4f1b-9343-f673788e1d66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:24.896964 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:24.915281 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9eff8-9409-4f1b-9343-f673788e1d66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:24.916768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9eff8-9409-4f1b-9343-f673788e1d66 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:24.943400 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93444b69-fdac-4e1b-8e7d-2ee43ecd2362 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:24.987474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b1a9d-5a24-48bb-abd9-06d97dc8b64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:24.988553 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b1a9d-5a24-48bb-abd9-06d97dc8b64b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:24.988844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b1a9d-5a24-48bb-abd9-06d97dc8b64b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:24.989128 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b1a9d-5a24-48bb-abd9-06d97dc8b64b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:24.998542 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b1a9d-5a24-48bb-abd9-06d97dc8b64b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:25.000494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b1a9d-5a24-48bb-abd9-06d97dc8b64b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:25.020486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-6d9b1a9d-5a24-48bb-abd9-06d97dc8b64b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:25.020486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:25.020486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:25.020486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:25.020486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:25.020486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:25.020486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:25.020486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:25.020486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:25.020486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:25.020486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:25.020486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:25.020486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:25.020486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:25.020486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:25.021650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:25.021650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:25.021650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:25.021650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:25.021650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:25.021650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:25.021650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:25.021650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:25.021650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:25.021650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:25.021650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:25.021650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:25.021650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:25.021650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:25.021650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:25.021650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:25.021650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:25.022824 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:25.022824 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:25.022824 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:25.022824 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:25.022824 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:25.022824 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:25.022824 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:25.022824 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:25.022824 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 264/530] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:25.022824 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b1a9d-5a24-48bb-abd9-06d97dc8b64b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:25.042364 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:25.058429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 267/531] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:21:25.058429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9eff8-9409-4f1b-9343-f673788e1d66 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:25.165349 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b1a9d-5a24-48bb-abd9-06d97dc8b64b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:25.194811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9eff8-9409-4f1b-9343-f673788e1d66 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:25.206548 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:25.215583 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c5f5e-c82c-49c7-b5d5-1be629309677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:25.216395 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c5f5e-c82c-49c7-b5d5-1be629309677 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:25.216682 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c5f5e-c82c-49c7-b5d5-1be629309677 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:25.216963 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c5f5e-c82c-49c7-b5d5-1be629309677 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:25.226689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c5f5e-c82c-49c7-b5d5-1be629309677 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:25.228483 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c5f5e-c82c-49c7-b5d5-1be629309677 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:25.244830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-1d6c5f5e-c82c-49c7-b5d5-1be629309677 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:25.244830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:25.244830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:25.244830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:25.244830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:25.244830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:25.244830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:25.244830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:25.244830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:25.244830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:25.244830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:25.244830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:25.244830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:25.244830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:25.244830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:25.245792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:25.245792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:25.245792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:25.245792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:25.245792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:25.245792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:25.245792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:25.245792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:25.245792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:25.245792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:25.245792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:25.245792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:25.245792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:25.245792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:25.245792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:25.245792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:25.245792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:25.246673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:25.246673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:25.246673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:25.246673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:25.246673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:25.246673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:25.246673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:25.246673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:25.246673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 265/532] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:25.246673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c5f5e-c82c-49c7-b5d5-1be629309677 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:25.252476 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97d46b-76d7-4db7-8450-771e1e18690a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:25.253189 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97d46b-76d7-4db7-8450-771e1e18690a None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:25.253517 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97d46b-76d7-4db7-8450-771e1e18690a None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:25.253805 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97d46b-76d7-4db7-8450-771e1e18690a None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:25.254657 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97d46b-76d7-4db7-8450-771e1e18690a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:25.254992 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97d46b-76d7-4db7-8450-771e1e18690a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:25.256446 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97d46b-76d7-4db7-8450-771e1e18690a None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:25.270100 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 268/533] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:25.271000 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97d46b-76d7-4db7-8450-771e1e18690a None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:25.332442 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:25.369487 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c5f5e-c82c-49c7-b5d5-1be629309677 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:25.384605 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97d46b-76d7-4db7-8450-771e1e18690a None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:25.405747 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd3613-dc8f-4aaf-80e3-f8453e6b13d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:25.406544 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd3613-dc8f-4aaf-80e3-f8453e6b13d7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:25.406834 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd3613-dc8f-4aaf-80e3-f8453e6b13d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:25.407217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd3613-dc8f-4aaf-80e3-f8453e6b13d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:25.418814 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd3613-dc8f-4aaf-80e3-f8453e6b13d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:25.420308 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd3613-dc8f-4aaf-80e3-f8453e6b13d7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:25.423174 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59193ea2-be77-4ed3-8237-81d39f396dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:25.424495 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59193ea2-be77-4ed3-8237-81d39f396dbe None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:25.424791 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59193ea2-be77-4ed3-8237-81d39f396dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:25.425075 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59193ea2-be77-4ed3-8237-81d39f396dbe None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:25.425920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59193ea2-be77-4ed3-8237-81d39f396dbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:25.426240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59193ea2-be77-4ed3-8237-81d39f396dbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:25.427736 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59193ea2-be77-4ed3-8237-81d39f396dbe None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:25.438832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-b2bd3613-dc8f-4aaf-80e3-f8453e6b13d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:25.438832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:25.438832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:25.438832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:25.438832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:25.438832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:25.438832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:25.438832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:25.438832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:25.438832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:25.438832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:25.438832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:25.438832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:25.438832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:25.438832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:25.440843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:25.440843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:25.440843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:25.440843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:25.440843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:25.440843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:25.440843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:25.440843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:25.440843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:25.440843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:25.440843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:25.440843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:25.440843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:25.440843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:25.440843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:25.440843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:25.440843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:25.442577 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:25.442577 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:25.442577 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:25.442577 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:25.442577 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 266/534] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:25.442577 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd3613-dc8f-4aaf-80e3-f8453e6b13d7 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:25.446177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 269/535] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:25.447064 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59193ea2-be77-4ed3-8237-81d39f396dbe None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:25.498203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:25.560841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd3613-dc8f-4aaf-80e3-f8453e6b13d7 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:25.570080 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59193ea2-be77-4ed3-8237-81d39f396dbe None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:25.594144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:25.598343 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c777d22-83a9-4b41-9822-733b1b62da79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:25.599018 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c777d22-83a9-4b41-9822-733b1b62da79 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:25.599309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c777d22-83a9-4b41-9822-733b1b62da79 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:25.599593 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c777d22-83a9-4b41-9822-733b1b62da79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:25.600443 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c777d22-83a9-4b41-9822-733b1b62da79 None admin] RBAC: Adding query filter params (role.id=0e34a6f9b89a4e29951fb189abacb34a, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=327b347d6295422c99ce7b44eab00467) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:25.600768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c777d22-83a9-4b41-9822-733b1b62da79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:25.602398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c777d22-83a9-4b41-9822-733b1b62da79 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:25.608442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 267/536] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:21:25 2022] GET /identity/v3/role_assignments?user.id=327b347d6295422c99ce7b44eab00467&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=0e34a6f9b89a4e29951fb189abacb34a => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:25.610247 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed63037-ea69-482d-a573-45b50a54be57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:25.611502 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed63037-ea69-482d-a573-45b50a54be57 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:25.611804 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed63037-ea69-482d-a573-45b50a54be57 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:25.612101 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed63037-ea69-482d-a573-45b50a54be57 None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:25.612864 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed63037-ea69-482d-a573-45b50a54be57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:25.613210 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed63037-ea69-482d-a573-45b50a54be57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:25.614664 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed63037-ea69-482d-a573-45b50a54be57 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:25.617516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c777d22-83a9-4b41-9822-733b1b62da79 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:25.630247 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 270/537] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:25.635984 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed63037-ea69-482d-a573-45b50a54be57 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:25.754789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed63037-ea69-482d-a573-45b50a54be57 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:25.794183 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050af19-9a10-4aef-a262-4e47a7b2dd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:25.795071 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050af19-9a10-4aef-a262-4e47a7b2dd1e None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:25.795369 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050af19-9a10-4aef-a262-4e47a7b2dd1e None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:25.795686 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050af19-9a10-4aef-a262-4e47a7b2dd1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:25.804599 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050af19-9a10-4aef-a262-4e47a7b2dd1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:25.806177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050af19-9a10-4aef-a262-4e47a7b2dd1e None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:25.824642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-0050af19-9a10-4aef-a262-4e47a7b2dd1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:25.824642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:25.824642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:25.824642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:25.824642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:25.824642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:25.824642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:25.824642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:25.824642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:25.824642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:25.824642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:25.824642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:25.824642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:25.824642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:25.824642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:25.826819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:25.826819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:25.826819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:25.826819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:25.826819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:25.826819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:25.826819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:25.826819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:25.826819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:25.826819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:25.826819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:25.826819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:25.826819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:25.826819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:25.826819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:25.826819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:25.826819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:25.829523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:25.829523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:25.829523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:25.829523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:25.829523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:25.829523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:25.829523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:25.829523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:25.830502 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c777d22-83a9-4b41-9822-733b1b62da79 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:25.830647 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 271/538] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:25.833496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050af19-9a10-4aef-a262-4e47a7b2dd1e None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:25.843561 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:25.877682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:25.895860 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7d89a-5325-4c29-aaef-2fe8ef7e68d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:25.896685 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7d89a-5325-4c29-aaef-2fe8ef7e68d2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:25.896947 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7d89a-5325-4c29-aaef-2fe8ef7e68d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:25.897284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7d89a-5325-4c29-aaef-2fe8ef7e68d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:25.898149 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a7d89a-5325-4c29-aaef-2fe8ef7e68d2 None admin] RBAC: Adding query filter params (role.id=0e34a6f9b89a4e29951fb189abacb34a, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=1fadecafbe2549acaa4cde5412638d8f) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:25.898470 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a7d89a-5325-4c29-aaef-2fe8ef7e68d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:25.900081 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a7d89a-5325-4c29-aaef-2fe8ef7e68d2 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:25.907052 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 268/539] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:21:25 2022] GET /identity/v3/role_assignments?user.id=1fadecafbe2549acaa4cde5412638d8f&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=0e34a6f9b89a4e29951fb189abacb34a => generated 259 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:25.908517 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f262219-9e7e-4be5-b468-42033f242ceb None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:25.908811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f262219-9e7e-4be5-b468-42033f242ceb None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:25.909094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f262219-9e7e-4be5-b468-42033f242ceb None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:25.910746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 269/540] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:25.916516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865c66f-5d3a-4007-8963-dce66f0d35a8 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:25.916815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865c66f-5d3a-4007-8963-dce66f0d35a8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:25.917224 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865c66f-5d3a-4007-8963-dce66f0d35a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:25.936059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a865c66f-5d3a-4007-8963-dce66f0d35a8 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:25.937344 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a865c66f-5d3a-4007-8963-dce66f0d35a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:25.997373 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050af19-9a10-4aef-a262-4e47a7b2dd1e None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:26.021092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 270/541] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:21:26.049984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a865c66f-5d3a-4007-8963-dce66f0d35a8 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:26.076562 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8080d-faeb-4ab2-a151-a7854134e512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:26.077284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8080d-faeb-4ab2-a151-a7854134e512 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:26.077597 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8080d-faeb-4ab2-a151-a7854134e512 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:26.077887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8080d-faeb-4ab2-a151-a7854134e512 None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:26.078715 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e8080d-faeb-4ab2-a151-a7854134e512 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:26.080780 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e8080d-faeb-4ab2-a151-a7854134e512 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:26.085208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e8080d-faeb-4ab2-a151-a7854134e512 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:26.108020 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 272/542] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:26.121439 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8080d-faeb-4ab2-a151-a7854134e512 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:26.224427 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:26.227228 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a865c66f-5d3a-4007-8963-dce66f0d35a8 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:26.261920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:26.270163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8080d-faeb-4ab2-a151-a7854134e512 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:26.300103 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2a867-413c-4dac-a978-bf431b0f68ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:26.301474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2a867-413c-4dac-a978-bf431b0f68ee None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:26.301989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2a867-413c-4dac-a978-bf431b0f68ee None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:26.302479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2a867-413c-4dac-a978-bf431b0f68ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:26.305647 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6cdd63-b1f9-41a0-af6b-fa97507ead73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:26.306559 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cdd63-b1f9-41a0-af6b-fa97507ead73 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:26.307069 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cdd63-b1f9-41a0-af6b-fa97507ead73 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:26.307565 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cdd63-b1f9-41a0-af6b-fa97507ead73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:26.308611 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6cdd63-b1f9-41a0-af6b-fa97507ead73 None admin] RBAC: Adding query filter params (role.id=0e34a6f9b89a4e29951fb189abacb34a, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=a303489493ce4cd2b62e876fbc8a7404) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:26.309132 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6cdd63-b1f9-41a0-af6b-fa97507ead73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:26.310819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6cdd63-b1f9-41a0-af6b-fa97507ead73 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:26.319727 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 273/543] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:21:26 2022] GET /identity/v3/role_assignments?user.id=a303489493ce4cd2b62e876fbc8a7404&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=0e34a6f9b89a4e29951fb189abacb34a => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:26.327301 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c2a867-413c-4dac-a978-bf431b0f68ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:26.336540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-e7c2a867-413c-4dac-a978-bf431b0f68ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:26.336540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:26.336540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:26.336540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:26.336540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:26.336540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:26.336540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:26.336540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:26.336540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:26.336540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:26.336540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:26.336540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:26.336540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:26.336540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:26.336540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:26.336540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:26.337567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:26.337567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:26.337567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:26.337567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:26.337567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:26.337567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:26.337567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:26.337567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:26.337567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:26.337567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:26.337567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:26.337567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:26.337567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:26.337567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:26.337567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:26.337567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:26.338387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:26.338387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:26.338387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:26.338387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:26.338387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:26.338387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:26.338387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:26.338387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:26.338387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:26.338387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:26.338387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:26.338387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:26.341258 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 271/544] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:21:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:26.350386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6cdd63-b1f9-41a0-af6b-fa97507ead73 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:26.510551 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6cdd63-b1f9-41a0-af6b-fa97507ead73 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:26.551966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e912af12-fa2a-409a-bf90-c8d54698d865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:26.553000 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912af12-fa2a-409a-bf90-c8d54698d865 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:26.553518 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912af12-fa2a-409a-bf90-c8d54698d865 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:26.554020 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912af12-fa2a-409a-bf90-c8d54698d865 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:26.554978 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e912af12-fa2a-409a-bf90-c8d54698d865 None admin] RBAC: Adding query filter params (name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:26.555529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e912af12-fa2a-409a-bf90-c8d54698d865 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:26.557114 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e912af12-fa2a-409a-bf90-c8d54698d865 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:26.567285 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 274/545] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 17:21:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:26.572541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2a867-413c-4dac-a978-bf431b0f68ee None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:26.750103 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2a867-413c-4dac-a978-bf431b0f68ee None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:26.771717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0232a9-f912-4c44-aef5-f1a4549788cc None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:26.772117 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0232a9-f912-4c44-aef5-f1a4549788cc None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:26.772590 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0232a9-f912-4c44-aef5-f1a4549788cc None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:26.780201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 275/546] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:21:26.781445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb3e0a-d6f9-4e36-a882-9d8ccc99bef9 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:26.782097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb3e0a-d6f9-4e36-a882-9d8ccc99bef9 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:26.782676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb3e0a-d6f9-4e36-a882-9d8ccc99bef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:26.807492 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-ceeb3e0a-d6f9-4e36-a882-9d8ccc99bef9 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:26.808782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-ceeb3e0a-d6f9-4e36-a882-9d8ccc99bef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:26.820921 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafec29-a08e-4e53-add1-733fc7a280b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:26.822113 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafec29-a08e-4e53-add1-733fc7a280b5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:26.822630 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafec29-a08e-4e53-add1-733fc7a280b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:26.823115 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafec29-a08e-4e53-add1-733fc7a280b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:26.832885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:26.847222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fafec29-a08e-4e53-add1-733fc7a280b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:26.849077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fafec29-a08e-4e53-add1-733fc7a280b5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:26.895457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-5fafec29-a08e-4e53-add1-733fc7a280b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:26.895457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:26.895457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:26.895457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:26.895457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:26.895457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:26.895457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:26.895457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:26.895457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:26.895457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:26.895457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:26.895457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:26.895457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:26.895457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:26.895457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:26.896425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:26.896425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:26.896425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:26.896425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:26.896425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:26.896425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:26.896425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:26.896425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:26.896425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:26.896425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:26.896425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:26.896425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:26.896425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:26.896425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:26.896425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:26.896425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:26.896425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:26.897752 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:26.897752 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:26.897752 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:26.897752 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:26.897752 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 272/547] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:26.904166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafec29-a08e-4e53-add1-733fc7a280b5 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:26.914363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 276/548] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:21:26.925545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:26.976004 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb3e0a-d6f9-4e36-a882-9d8ccc99bef9 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:27.021341 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafec29-a08e-4e53-add1-733fc7a280b5 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:27.073179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b07b5d-f343-4c1f-8d10-e7e848fd3137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:27.073179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b07b5d-f343-4c1f-8d10-e7e848fd3137 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:27.073179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b07b5d-f343-4c1f-8d10-e7e848fd3137 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:27.073179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b07b5d-f343-4c1f-8d10-e7e848fd3137 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:27.073610 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b07b5d-f343-4c1f-8d10-e7e848fd3137 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:27.073610 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b07b5d-f343-4c1f-8d10-e7e848fd3137 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:27.073610 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b07b5d-f343-4c1f-8d10-e7e848fd3137 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:27.097422 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 273/549] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:27.125285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b07b5d-f343-4c1f-8d10-e7e848fd3137 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:27.162919 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb3e0a-d6f9-4e36-a882-9d8ccc99bef9 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:27.231135 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e783b7-2c54-43ec-9e68-b46edfe63920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:27.232456 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e783b7-2c54-43ec-9e68-b46edfe63920 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:27.233105 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e783b7-2c54-43ec-9e68-b46edfe63920 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:27.234032 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:27.234863 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e783b7-2c54-43ec-9e68-b46edfe63920 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:27.277198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e783b7-2c54-43ec-9e68-b46edfe63920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:27.277817 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b07b5d-f343-4c1f-8d10-e7e848fd3137 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:27.299403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-f3e783b7-2c54-43ec-9e68-b46edfe63920 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:27.299403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:27.299403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:27.299403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:27.299403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:27.299403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:27.299403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:27.299403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:27.299403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:27.299403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:27.299403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:27.299403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:27.299403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:27.299403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:27.299403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:27.299403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:27.301420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:27.301420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:27.301420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:27.301420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:27.301420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:27.301420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:27.301420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:27.301420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:27.301420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:27.301420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:27.301420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:27.301420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:27.301420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:27.301420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:27.301420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:27.301420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:27.303116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:27.303116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:27.303116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:27.303116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:27.303116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:27.303116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:27.303116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:27.303116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:27.303116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:27.303116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:27.303116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:27.303116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:27.307963 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 277/550] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:21:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:27.311644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b822b9-2e6d-4aeb-9479-2a8c327f08f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:27.312241 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b822b9-2e6d-4aeb-9479-2a8c327f08f0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:27.312531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b822b9-2e6d-4aeb-9479-2a8c327f08f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:27.313569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b822b9-2e6d-4aeb-9479-2a8c327f08f0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:27.314621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e783b7-2c54-43ec-9e68-b46edfe63920 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:27.321771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b822b9-2e6d-4aeb-9479-2a8c327f08f0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:27.323316 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b822b9-2e6d-4aeb-9479-2a8c327f08f0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:27.352641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-f6b822b9-2e6d-4aeb-9479-2a8c327f08f0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:21:27.352641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:27.352641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:27.352641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:27.352641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:27.352641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:27.352641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:27.352641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:27.352641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:27.352641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:27.352641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:27.352641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:27.352641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:27.352641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:27.352641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:27.352641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:27.353700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:27.353700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:27.353700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:27.353700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:27.353700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:27.353700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:27.353700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:27.353700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:27.353700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:27.353700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:27.353700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:27.353700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:27.353700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:27.353700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:27.353700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:27.353700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:27.353700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:27.353700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:27.354821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:27.354821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:27.354821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:27.354821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:27.354821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:27.354821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:27.354821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:27.354821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:27.354821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:21:27.354821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:27.361561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:27.364404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 274/551] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:21:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:27.370997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b822b9-2e6d-4aeb-9479-2a8c327f08f0 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:27.572995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e783b7-2c54-43ec-9e68-b46edfe63920 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:27.598447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b822b9-2e6d-4aeb-9479-2a8c327f08f0 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:27.633232 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:27.641883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c300c5-c0bb-4acf-9cb6-bcd893adb0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:27.642870 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c300c5-c0bb-4acf-9cb6-bcd893adb0db None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:27.643363 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c300c5-c0bb-4acf-9cb6-bcd893adb0db None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:27.643854 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c300c5-c0bb-4acf-9cb6-bcd893adb0db None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:27.644904 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c300c5-c0bb-4acf-9cb6-bcd893adb0db None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:27.645445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c300c5-c0bb-4acf-9cb6-bcd893adb0db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:27.647086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c300c5-c0bb-4acf-9cb6-bcd893adb0db None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:27.654887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb06fc-8ff4-47bd-8200-5d749f9acd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:27.655860 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb06fc-8ff4-47bd-8200-5d749f9acd95 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:27.656360 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb06fc-8ff4-47bd-8200-5d749f9acd95 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:27.656929 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb06fc-8ff4-47bd-8200-5d749f9acd95 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:27.658587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb06fc-8ff4-47bd-8200-5d749f9acd95 None admin] RBAC: Adding query filter params (name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:27.659332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb06fc-8ff4-47bd-8200-5d749f9acd95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:27.662332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb06fc-8ff4-47bd-8200-5d749f9acd95 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:27.680792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 275/553] 10.222.0.40 () {56 vars in 1135 bytes} [Thu Jun 16 17:21:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:27.681114 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 278/553] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 17:21:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:27.686724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c300c5-c0bb-4acf-9cb6-bcd893adb0db None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:27.688370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb06fc-8ff4-47bd-8200-5d749f9acd95 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:27.777186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:27.805055 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c300c5-c0bb-4acf-9cb6-bcd893adb0db None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:27.812128 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb06fc-8ff4-47bd-8200-5d749f9acd95 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:27.841206 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43667e-4145-44a1-a463-8a238a218b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:27.841672 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43667e-4145-44a1-a463-8a238a218b0a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:27.842170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43667e-4145-44a1-a463-8a238a218b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:27.842868 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43667e-4145-44a1-a463-8a238a218b0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:27.854797 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43667e-4145-44a1-a463-8a238a218b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:27.856554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43667e-4145-44a1-a463-8a238a218b0a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:27.865826 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0a6af-05fd-49f0-8e43-5b22eb0a3a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:27.866609 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0a6af-05fd-49f0-8e43-5b22eb0a3a04 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:27.866906 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0a6af-05fd-49f0-8e43-5b22eb0a3a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:27.867192 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0a6af-05fd-49f0-8e43-5b22eb0a3a04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:27.878172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-9a43667e-4145-44a1-a463-8a238a218b0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:27.878172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:27.878172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:27.878172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:27.878172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:27.878172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:27.878172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:27.878172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:27.878172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:27.878172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:27.878172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:27.878172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:27.878172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:27.878172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:27.878172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:27.879101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:27.879101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:27.879101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:27.879101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:27.879101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:27.879101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:27.879101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:27.879101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:27.879101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:27.879101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:27.879101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:27.879101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:27.879101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:27.879101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:27.879101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:27.879101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:27.879101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:27.879992 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:27.879992 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:27.879992 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:27.879992 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:27.882287 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 276/554] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:27.893276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43667e-4145-44a1-a463-8a238a218b0a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:27.953926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b0a6af-05fd-49f0-8e43-5b22eb0a3a04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:27.955402 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b0a6af-05fd-49f0-8e43-5b22eb0a3a04 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:27.977680 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:28.037909 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43667e-4145-44a1-a463-8a238a218b0a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:28.054128 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:28.074118 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79e9a7-d5fa-40da-b0ad-38527febb8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:28.075018 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79e9a7-d5fa-40da-b0ad-38527febb8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:28.075374 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79e9a7-d5fa-40da-b0ad-38527febb8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:28.075723 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79e9a7-d5fa-40da-b0ad-38527febb8d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:28.076558 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79e9a7-d5fa-40da-b0ad-38527febb8d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:28.076910 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79e9a7-d5fa-40da-b0ad-38527febb8d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:28.078404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79e9a7-d5fa-40da-b0ad-38527febb8d6 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:28.089449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 279/555] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:21:28.095146 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 277/556] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:28.100273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0a6af-05fd-49f0-8e43-5b22eb0a3a04 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:28.100460 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79e9a7-d5fa-40da-b0ad-38527febb8d6 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:28.214545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0a6af-05fd-49f0-8e43-5b22eb0a3a04 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:28.214721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79e9a7-d5fa-40da-b0ad-38527febb8d6 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:28.216239 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:28.263398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473c7e6-2b90-4bad-b47d-1e2b8f0027e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:28.264550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473c7e6-2b90-4bad-b47d-1e2b8f0027e7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:28.265039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473c7e6-2b90-4bad-b47d-1e2b8f0027e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:28.265554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473c7e6-2b90-4bad-b47d-1e2b8f0027e7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:28.268256 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc6d6a-5acd-4af2-85ce-1a41dc6c67d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:28.269193 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc6d6a-5acd-4af2-85ce-1a41dc6c67d4 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:28.269470 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc6d6a-5acd-4af2-85ce-1a41dc6c67d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:28.269857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc6d6a-5acd-4af2-85ce-1a41dc6c67d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:28.275241 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473c7e6-2b90-4bad-b47d-1e2b8f0027e7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:28.276931 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473c7e6-2b90-4bad-b47d-1e2b8f0027e7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:28.291934 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc6d6a-5acd-4af2-85ce-1a41dc6c67d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:28.293303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-e473c7e6-2b90-4bad-b47d-1e2b8f0027e7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:21:28.293303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:28.293303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:28.293303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:28.293303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:28.293303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:28.293303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:28.293303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:28.293303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:28.293303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:28.293303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:28.293303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:28.293303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:28.293303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:28.293303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:28.293303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:28.294898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:28.294898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:28.294898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:28.294898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:28.294898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:28.294898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:28.294898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:28.294898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:28.294898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:28.294898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:28.294898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:28.294898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:28.294898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:28.294898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:28.294898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:28.294898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:28.294898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:28.294898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:28.296099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:28.296099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:28.296099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:28.296099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:28.296099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:28.296099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:28.296099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:28.296099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:28.296099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:21:28.296099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:28.296789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc6d6a-5acd-4af2-85ce-1a41dc6c67d4 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:28.298582 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 278/557] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jun 16 17:21:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:21:28.310716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473c7e6-2b90-4bad-b47d-1e2b8f0027e7 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:28.320104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-00fc6d6a-5acd-4af2-85ce-1a41dc6c67d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:28.320104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:28.320104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:28.320104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:28.320104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:28.320104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:28.320104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:28.320104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:28.320104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:28.320104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:28.320104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:28.320104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:28.320104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:28.320104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:28.320104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:28.321254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:28.321254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:28.321254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:28.321254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:28.321254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:28.321254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:28.321254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:28.321254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:28.321254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:28.321254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:28.321254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:28.321254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:28.321254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:28.321254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:28.321254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:28.321254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:28.321254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:28.322305 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:28.322305 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:28.322305 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:28.322305 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:28.324247 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 280/558] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:28.334493 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc6d6a-5acd-4af2-85ce-1a41dc6c67d4 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:28.381303 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:28.428921 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473c7e6-2b90-4bad-b47d-1e2b8f0027e7 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:28.471550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dd0df-04da-4197-a9c6-da7db3959125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:28.472157 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dd0df-04da-4197-a9c6-da7db3959125 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:28.472447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dd0df-04da-4197-a9c6-da7db3959125 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:28.472792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dd0df-04da-4197-a9c6-da7db3959125 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:28.473726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dd0df-04da-4197-a9c6-da7db3959125 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:28.474100 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dd0df-04da-4197-a9c6-da7db3959125 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:28.475584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dd0df-04da-4197-a9c6-da7db3959125 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:28.495108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 279/559] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jun 16 17:21:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:28.510447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dd0df-04da-4197-a9c6-da7db3959125 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:28.539380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:28.551117 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc6d6a-5acd-4af2-85ce-1a41dc6c67d4 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:28.596753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaea7cd-336c-4adc-ac57-56a0e7be7c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:28.597454 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaea7cd-336c-4adc-ac57-56a0e7be7c33 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:28.597741 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaea7cd-336c-4adc-ac57-56a0e7be7c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:28.598020 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaea7cd-336c-4adc-ac57-56a0e7be7c33 None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:28.598755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaea7cd-336c-4adc-ac57-56a0e7be7c33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:28.599068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaea7cd-336c-4adc-ac57-56a0e7be7c33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:28.600472 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaea7cd-336c-4adc-ac57-56a0e7be7c33 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:28.617378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 281/560] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:28.619007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861e41f-4a37-406b-b5f8-1e8de6364396 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:28.619549 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861e41f-4a37-406b-b5f8-1e8de6364396 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:28.620014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861e41f-4a37-406b-b5f8-1e8de6364396 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:28.621834 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 282/561] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:28.625004 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861e41f-4a37-406b-b5f8-1e8de6364396 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:28.636317 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dd0df-04da-4197-a9c6-da7db3959125 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:28.681984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b066777-87bd-47fb-94ab-d82451a4b635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:28.683119 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b066777-87bd-47fb-94ab-d82451a4b635 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:28.683590 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b066777-87bd-47fb-94ab-d82451a4b635 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:28.684104 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b066777-87bd-47fb-94ab-d82451a4b635 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:28.696575 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b066777-87bd-47fb-94ab-d82451a4b635 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:28.698652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b066777-87bd-47fb-94ab-d82451a4b635 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:28.721511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-8b066777-87bd-47fb-94ab-d82451a4b635 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:28.721511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:28.721511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:28.721511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:28.721511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:28.721511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:28.721511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:28.721511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:28.721511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:28.721511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:28.721511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:28.721511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:28.721511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:28.721511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:28.721511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:28.722623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:28.722623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:28.722623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:28.722623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:28.722623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:28.722623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:28.722623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:28.722623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:28.722623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:28.722623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:28.722623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:28.722623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:28.722623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:28.722623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:28.722623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:28.722623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:28.722623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:28.723561 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:28.723561 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:28.723561 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:28.723561 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:28.726286 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 280/562] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:21:28.727955 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8302b-52d3-454b-bf12-8c573268e781 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:28.728441 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8302b-52d3-454b-bf12-8c573268e781 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:28.728902 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8302b-52d3-454b-bf12-8c573268e781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:28.747232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:28.749273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e7c8302b-52d3-454b-bf12-8c573268e781 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:28.750880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e7c8302b-52d3-454b-bf12-8c573268e781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:28.767492 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861e41f-4a37-406b-b5f8-1e8de6364396 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:28.805061 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b031e-285f-4e75-8d21-932784857bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:28.805898 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b031e-285f-4e75-8d21-932784857bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:28.806186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b031e-285f-4e75-8d21-932784857bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:28.806471 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b031e-285f-4e75-8d21-932784857bd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:28.814015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b031e-285f-4e75-8d21-932784857bd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:28.815471 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b031e-285f-4e75-8d21-932784857bd6 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:28.827152 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 281/563] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:28.828264 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8302b-52d3-454b-bf12-8c573268e781 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:28.830529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-2d2b031e-285f-4e75-8d21-932784857bd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:28.830529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:28.830529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:28.830529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:28.830529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:28.830529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:28.830529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:28.830529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:28.830529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:28.830529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:28.830529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:28.830529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:28.830529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:28.830529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:28.830529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:28.831452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:28.831452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:28.831452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:28.831452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:28.831452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:28.831452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:28.831452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:28.831452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:28.831452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:28.831452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:28.831452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:28.831452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:28.831452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:28.831452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:28.831452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:28.831452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:28.831452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:28.832370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:28.832370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:28.832370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:28.832370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:28.832370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:28.832370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:28.832370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:28.832370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:28.834670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:28.836687 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 283/564] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:28.838242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a95a9-584e-44a4-b68c-d4f097993412 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:28.838671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a95a9-584e-44a4-b68c-d4f097993412 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:28.838832 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a95a9-584e-44a4-b68c-d4f097993412 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:28.840346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 284/565] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:28.841047 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a95a9-584e-44a4-b68c-d4f097993412 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:28.961694 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8302b-52d3-454b-bf12-8c573268e781 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:28.963935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a95a9-584e-44a4-b68c-d4f097993412 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:29.000459 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed71dff-235f-417c-9193-048f36a2af5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:29.001960 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed71dff-235f-417c-9193-048f36a2af5c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:29.002252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed71dff-235f-417c-9193-048f36a2af5c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:29.002531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed71dff-235f-417c-9193-048f36a2af5c None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:29.003271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed71dff-235f-417c-9193-048f36a2af5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:29.003582 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed71dff-235f-417c-9193-048f36a2af5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:29.004902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140faea5-38ea-4be6-b7a0-60666777d339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:29.005024 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed71dff-235f-417c-9193-048f36a2af5c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:29.005598 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140faea5-38ea-4be6-b7a0-60666777d339 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:29.005892 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140faea5-38ea-4be6-b7a0-60666777d339 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:29.006172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140faea5-38ea-4be6-b7a0-60666777d339 None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:29.007641 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140faea5-38ea-4be6-b7a0-60666777d339 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:29.007976 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140faea5-38ea-4be6-b7a0-60666777d339 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:29.009734 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140faea5-38ea-4be6-b7a0-60666777d339 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:29.022574 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 282/566] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:29.022574 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff4540-a454-477f-9840-a00cbae9d6ef None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:29.022574 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff4540-a454-477f-9840-a00cbae9d6ef None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:29.022574 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff4540-a454-477f-9840-a00cbae9d6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:29.028684 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 285/567] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:29.029567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140faea5-38ea-4be6-b7a0-60666777d339 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:29.037260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:29.040541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-58ff4540-a454-477f-9840-a00cbae9d6ef None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:29.041784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-58ff4540-a454-477f-9840-a00cbae9d6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:29.105244 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:29.120348 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 283/568] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:29.121488 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff4540-a454-477f-9840-a00cbae9d6ef None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:29.155089 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140faea5-38ea-4be6-b7a0-60666777d339 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:29.195188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84174671-ec60-469c-b0c3-3c1dd0cffe92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:29.196005 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84174671-ec60-469c-b0c3-3c1dd0cffe92 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:29.196298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84174671-ec60-469c-b0c3-3c1dd0cffe92 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:29.196585 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84174671-ec60-469c-b0c3-3c1dd0cffe92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:29.208067 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84174671-ec60-469c-b0c3-3c1dd0cffe92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:29.215957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-84174671-ec60-469c-b0c3-3c1dd0cffe92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:29.215957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:29.215957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:29.215957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:29.215957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:29.215957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:29.215957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:29.215957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:29.215957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:29.215957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:29.215957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:29.215957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:29.215957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:29.215957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:29.215957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:29.215957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:29.217076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:29.217076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:29.217076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:29.217076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:29.217076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:29.217076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:29.217076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:29.217076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:29.217076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:29.217076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:29.217076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:29.217076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:29.217076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:29.217076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:29.217076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:29.217076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:29.217986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:29.217986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:29.217986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:29.217986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:29.217986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:29.217986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:29.217986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:29.217986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:29.217986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:29.217986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:29.217986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:29.217986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:29.217986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 286/569] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:21:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:29.217986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84174671-ec60-469c-b0c3-3c1dd0cffe92 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:29.245308 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff4540-a454-477f-9840-a00cbae9d6ef None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:29.280150 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61304a2-82b4-4be1-9f6b-724414b75547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:29.280956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61304a2-82b4-4be1-9f6b-724414b75547 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:29.281333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61304a2-82b4-4be1-9f6b-724414b75547 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:29.281550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61304a2-82b4-4be1-9f6b-724414b75547 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:29.282828 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:29.289033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61304a2-82b4-4be1-9f6b-724414b75547 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:29.290513 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61304a2-82b4-4be1-9f6b-724414b75547 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:29.304839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-d61304a2-82b4-4be1-9f6b-724414b75547 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:29.304839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:29.304839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:29.304839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:29.304839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:29.304839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:29.304839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:29.304839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:29.304839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:29.304839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:29.304839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:29.304839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:29.304839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:29.304839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:29.304839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:29.306031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:29.306031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:29.306031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:29.306031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:29.306031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:29.306031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:29.306031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:29.306031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:29.306031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:29.306031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:29.306031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:29.306031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:29.306031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:29.306031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:29.306031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:29.306031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:29.306031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:29.311637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:29.311637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:29.311637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:29.311637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:29.311637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:29.311637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:29.311637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:29.311637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:29.311637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 284/570] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:29.311637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61304a2-82b4-4be1-9f6b-724414b75547 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:29.322289 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84174671-ec60-469c-b0c3-3c1dd0cffe92 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:29.356799 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c7914-1d0e-4985-8929-a7b981174472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:29.357486 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c7914-1d0e-4985-8929-a7b981174472 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:29.357780 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c7914-1d0e-4985-8929-a7b981174472 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:29.358063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c7914-1d0e-4985-8929-a7b981174472 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:29.358902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c7914-1d0e-4985-8929-a7b981174472 None admin] RBAC: Adding query filter params (role.id=0e34a6f9b89a4e29951fb189abacb34a, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=b8a37d1201d54afdad9b228ef4f2d787) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:29.359219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c7914-1d0e-4985-8929-a7b981174472 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:29.359895 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:29.361222 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c7914-1d0e-4985-8929-a7b981174472 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:29.366896 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 287/571] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:21:29 2022] GET /identity/v3/role_assignments?user.id=b8a37d1201d54afdad9b228ef4f2d787&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=0e34a6f9b89a4e29951fb189abacb34a => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:29.367948 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28db5b-8fd3-4f75-b9bb-f815c1781891 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:29.368278 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28db5b-8fd3-4f75-b9bb-f815c1781891 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:29.368523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28db5b-8fd3-4f75-b9bb-f815c1781891 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:29.369812 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 288/572] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:29.370532 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb28db5b-8fd3-4f75-b9bb-f815c1781891 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:29.419492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61304a2-82b4-4be1-9f6b-724414b75547 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:29.459440 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a1d6c-f01a-4d60-af6d-1e0e0db2fa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:29.460272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a1d6c-f01a-4d60-af6d-1e0e0db2fa7e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:29.460566 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a1d6c-f01a-4d60-af6d-1e0e0db2fa7e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:29.460942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a1d6c-f01a-4d60-af6d-1e0e0db2fa7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:29.473739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a1d6c-f01a-4d60-af6d-1e0e0db2fa7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:29.481874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-ff4a1d6c-f01a-4d60-af6d-1e0e0db2fa7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:29.481874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:29.481874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:29.481874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:29.481874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:29.481874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:29.481874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:29.481874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:29.481874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:29.481874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:29.481874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:29.481874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:29.481874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:29.481874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:29.481874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:29.481874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:29.483272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:29.483272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:29.483272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:29.483272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:29.483272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:29.483272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:29.483272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:29.483272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:29.483272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:29.483272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:29.483272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:29.483272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:29.483272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:29.483272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:29.483272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:29.483272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:29.484327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:29.484327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:29.484327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:29.484327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:29.484327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:29.484327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:29.484327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:29.484327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:29.484327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:29.484327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:29.484327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:29.484327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:29.489678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 285/573] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:21:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:29.490501 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a1d6c-f01a-4d60-af6d-1e0e0db2fa7e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:29.500401 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb28db5b-8fd3-4f75-b9bb-f815c1781891 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:29.532332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:29.541519 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78647a70-d009-4aaa-b506-b47b661e6616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:29.542202 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78647a70-d009-4aaa-b506-b47b661e6616 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:29.542491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78647a70-d009-4aaa-b506-b47b661e6616 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:29.542775 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78647a70-d009-4aaa-b506-b47b661e6616 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:29.543535 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78647a70-d009-4aaa-b506-b47b661e6616 None admin] RBAC: Adding query filter params (name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:29.543847 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78647a70-d009-4aaa-b506-b47b661e6616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:29.545251 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78647a70-d009-4aaa-b506-b47b661e6616 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:29.554199 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 289/574] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 17:21:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:29.555434 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771f6ca-d545-49e5-8539-cb0a05402306 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:29.555720 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771f6ca-d545-49e5-8539-cb0a05402306 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:29.555999 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771f6ca-d545-49e5-8539-cb0a05402306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:29.572149 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-6771f6ca-d545-49e5-8539-cb0a05402306 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:29.573378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-6771f6ca-d545-49e5-8539-cb0a05402306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:29.618603 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a1d6c-f01a-4d60-af6d-1e0e0db2fa7e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:29.632875 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:29.647332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 290/575] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:29.648005 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771f6ca-d545-49e5-8539-cb0a05402306 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:29.669928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01691c14-fc3a-4835-a60d-dacd6769d810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:29.670612 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01691c14-fc3a-4835-a60d-dacd6769d810 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:29.671468 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01691c14-fc3a-4835-a60d-dacd6769d810 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:29.671756 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01691c14-fc3a-4835-a60d-dacd6769d810 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:29.673121 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01691c14-fc3a-4835-a60d-dacd6769d810 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:29.673467 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01691c14-fc3a-4835-a60d-dacd6769d810 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:29.674923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01691c14-fc3a-4835-a60d-dacd6769d810 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:29.694266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 286/576] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:29.694266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01691c14-fc3a-4835-a60d-dacd6769d810 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:29.763558 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771f6ca-d545-49e5-8539-cb0a05402306 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:29.788496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:29.804399 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe293af-182a-41c3-8d70-e5c6a67c5930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:29.805076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe293af-182a-41c3-8d70-e5c6a67c5930 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:29.805393 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe293af-182a-41c3-8d70-e5c6a67c5930 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:29.805672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe293af-182a-41c3-8d70-e5c6a67c5930 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:29.806092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01691c14-fc3a-4835-a60d-dacd6769d810 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:29.806423 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe293af-182a-41c3-8d70-e5c6a67c5930 None admin] RBAC: Adding query filter params (name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:29.806733 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe293af-182a-41c3-8d70-e5c6a67c5930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:29.808097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe293af-182a-41c3-8d70-e5c6a67c5930 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:29.817601 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 291/577] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 17:21:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:29.822940 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe293af-182a-41c3-8d70-e5c6a67c5930 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:29.846203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900dfa6d-7724-44d9-bada-18e2f169ab14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:29.847019 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dfa6d-7724-44d9-bada-18e2f169ab14 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:29.847314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dfa6d-7724-44d9-bada-18e2f169ab14 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:29.847605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dfa6d-7724-44d9-bada-18e2f169ab14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:29.858661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900dfa6d-7724-44d9-bada-18e2f169ab14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:29.860170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900dfa6d-7724-44d9-bada-18e2f169ab14 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:29.880269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-900dfa6d-7724-44d9-bada-18e2f169ab14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:29.880269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:29.880269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:29.880269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:29.880269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:29.880269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:29.880269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:29.880269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:29.880269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:29.880269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:29.880269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:29.880269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:29.880269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:29.880269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:29.880269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:29.881269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:29.881269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:29.881269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:29.881269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:29.881269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:29.881269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:29.881269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:29.881269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:29.881269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:29.881269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:29.881269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:29.881269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:29.881269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:29.881269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:29.881269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:29.881269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:29.881269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:29.882416 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:29.882416 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:29.882416 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:29.882416 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:29.886266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 287/578] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:29.887138 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900dfa6d-7724-44d9-bada-18e2f169ab14 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:29.916430 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:29.936023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe293af-182a-41c3-8d70-e5c6a67c5930 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:29.973688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d3bf5c-ebc4-40de-a3d0-2c668a3cd5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:29.974487 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3bf5c-ebc4-40de-a3d0-2c668a3cd5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:29.974808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3bf5c-ebc4-40de-a3d0-2c668a3cd5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:29.975073 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3bf5c-ebc4-40de-a3d0-2c668a3cd5b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:29.987114 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d3bf5c-ebc4-40de-a3d0-2c668a3cd5b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:29.994889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-34d3bf5c-ebc4-40de-a3d0-2c668a3cd5b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:29.994889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:29.994889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:29.994889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:29.994889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:29.994889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:29.994889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:29.994889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:29.994889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:29.994889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:29.994889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:29.994889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:29.994889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:29.994889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:29.994889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:29.994889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:29.995879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:29.995879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:29.995879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:29.995879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:29.995879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:29.995879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:29.995879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:29.995879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:29.995879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:29.995879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:29.995879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:29.995879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:29.995879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:29.995879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:29.995879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:29.995879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:30.001872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:30.001872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:30.001872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:30.001872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:30.001872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:30.001872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:30.001872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:30.001872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:30.001872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:30.001872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:30.001872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:30.001872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:30.002724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900dfa6d-7724-44d9-bada-18e2f169ab14 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:30.002817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 292/579] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:21:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:30.002817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d3bf5c-ebc4-40de-a3d0-2c668a3cd5b4 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:30.042093 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2683b-2f3d-4245-88e8-23795602ca25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:30.042929 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2683b-2f3d-4245-88e8-23795602ca25 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:30.043216 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2683b-2f3d-4245-88e8-23795602ca25 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:30.043495 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2683b-2f3d-4245-88e8-23795602ca25 None admin] PATH_INFO: `/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/689a31e26c6b454a91c3fd79678b4d07/roles/0e34a6f9b89a4e29951fb189abacb34a` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:30.053217 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:30.059555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2683b-2f3d-4245-88e8-23795602ca25 None admin] RBAC: Authorizing `identity:create_grant(project_id=35a535a305e740368b7a4d77d0ad22a0, user_id=689a31e26c6b454a91c3fd79678b4d07, role_id=0e34a6f9b89a4e29951fb189abacb34a)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:30.061015 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2683b-2f3d-4245-88e8-23795602ca25 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:30.085183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 288/580] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:21:29 2022] PUT /identity/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/689a31e26c6b454a91c3fd79678b4d07/roles/0e34a6f9b89a4e29951fb189abacb34a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:21:30.085183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2683b-2f3d-4245-88e8-23795602ca25 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:30.150301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d3bf5c-ebc4-40de-a3d0-2c668a3cd5b4 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:30.189061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:30.205095 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fc05b-d8a6-491b-8449-eacda1e73151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:30.206826 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fc05b-d8a6-491b-8449-eacda1e73151 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:30.207327 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fc05b-d8a6-491b-8449-eacda1e73151 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:30.207813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fc05b-d8a6-491b-8449-eacda1e73151 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:30.223933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fc05b-d8a6-491b-8449-eacda1e73151 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:30.226554 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fc05b-d8a6-491b-8449-eacda1e73151 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:30.229369 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2683b-2f3d-4245-88e8-23795602ca25 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:30.251707 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-481fc05b-d8a6-491b-8449-eacda1e73151 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:30.251707 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:30.251707 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:30.251707 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:30.251707 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:30.251707 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:30.251707 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:30.251707 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:30.251707 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:30.251707 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:30.251707 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:30.251707 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:30.251707 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:30.251707 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:30.251707 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:30.252670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:30.252670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:30.252670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:30.252670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:30.252670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:30.252670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:30.252670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:30.252670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:30.252670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:30.252670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:30.252670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:30.252670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:30.252670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:30.252670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:30.252670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:30.252670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:30.252670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:30.253613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:30.253613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:30.253613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:30.253613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:30.256063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 293/581] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:30.256063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fc05b-d8a6-491b-8449-eacda1e73151 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:30.278272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f46074-dbe9-4be0-8d04-bfa8ae7e78be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:30.279047 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f46074-dbe9-4be0-8d04-bfa8ae7e78be None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:30.279309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f46074-dbe9-4be0-8d04-bfa8ae7e78be None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:30.279648 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f46074-dbe9-4be0-8d04-bfa8ae7e78be None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:30.280390 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f46074-dbe9-4be0-8d04-bfa8ae7e78be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:30.280713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f46074-dbe9-4be0-8d04-bfa8ae7e78be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:30.282201 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f46074-dbe9-4be0-8d04-bfa8ae7e78be None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:30.295738 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 289/582] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:30.296521 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f46074-dbe9-4be0-8d04-bfa8ae7e78be None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:30.380415 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:30.392736 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fc05b-d8a6-491b-8449-eacda1e73151 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:30.428819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db15d0-76e2-4bd0-a7fe-64a6bc91c281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:30.429554 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db15d0-76e2-4bd0-a7fe-64a6bc91c281 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:30.429851 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db15d0-76e2-4bd0-a7fe-64a6bc91c281 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:30.430209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db15d0-76e2-4bd0-a7fe-64a6bc91c281 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:30.430898 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db15d0-76e2-4bd0-a7fe-64a6bc91c281 None admin] RBAC: Adding query filter params (name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:30.431217 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db15d0-76e2-4bd0-a7fe-64a6bc91c281 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:30.432608 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db15d0-76e2-4bd0-a7fe-64a6bc91c281 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:30.433098 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f46074-dbe9-4be0-8d04-bfa8ae7e78be None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:30.446721 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 294/583] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 17:21:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:30.447649 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db15d0-76e2-4bd0-a7fe-64a6bc91c281 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:30.474894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def8f52-0745-4fcd-80b0-aa5ce55fad70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:30.476113 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def8f52-0745-4fcd-80b0-aa5ce55fad70 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:30.476612 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def8f52-0745-4fcd-80b0-aa5ce55fad70 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:30.477351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def8f52-0745-4fcd-80b0-aa5ce55fad70 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:30.479381 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def8f52-0745-4fcd-80b0-aa5ce55fad70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:30.479955 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def8f52-0745-4fcd-80b0-aa5ce55fad70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:30.481601 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def8f52-0745-4fcd-80b0-aa5ce55fad70 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:30.483689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:30.498191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 290/584] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:30.499384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def8f52-0745-4fcd-80b0-aa5ce55fad70 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:30.572573 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db15d0-76e2-4bd0-a7fe-64a6bc91c281 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:30.609215 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4889ba-286a-48cf-8029-e07a46644298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:30.609608 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4889ba-286a-48cf-8029-e07a46644298 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:30.610319 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4889ba-286a-48cf-8029-e07a46644298 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:30.610664 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4889ba-286a-48cf-8029-e07a46644298 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:30.616284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def8f52-0745-4fcd-80b0-aa5ce55fad70 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:30.619798 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4889ba-286a-48cf-8029-e07a46644298 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:30.621299 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4889ba-286a-48cf-8029-e07a46644298 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:30.637496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-5f4889ba-286a-48cf-8029-e07a46644298 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:21:30.637496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:30.637496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:30.637496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:30.637496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:30.637496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:30.637496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:30.637496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:30.637496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:30.637496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:30.637496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:30.637496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:30.637496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:30.637496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:30.637496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:30.637496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:30.638561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:30.638561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:30.638561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:30.638561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:30.638561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:30.638561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:30.638561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:30.638561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:30.638561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:30.638561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:30.638561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:30.638561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:30.638561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:30.638561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:30.638561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:30.638561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:30.638561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:30.638561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:30.639574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:30.639574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:30.639574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:30.639574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:30.639574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:30.639574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:30.639574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:30.639574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:30.639574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:21:30.639574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:30.643344 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 295/585] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:30.644346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4889ba-286a-48cf-8029-e07a46644298 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:30.658848 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d5801f-9516-4e80-a651-7432ddbaf0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:30.659985 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5801f-9516-4e80-a651-7432ddbaf0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:30.660527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5801f-9516-4e80-a651-7432ddbaf0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:30.661125 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5801f-9516-4e80-a651-7432ddbaf0e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:30.664014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:30.675796 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d5801f-9516-4e80-a651-7432ddbaf0e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:30.678069 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d5801f-9516-4e80-a651-7432ddbaf0e5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:30.698404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-a4d5801f-9516-4e80-a651-7432ddbaf0e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:30.698404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:30.698404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:30.698404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:30.698404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:30.698404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:30.698404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:30.698404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:30.698404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:30.698404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:30.698404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:30.698404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:30.698404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:30.698404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:30.698404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:30.699378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:30.699378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:30.699378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:30.699378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:30.699378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:30.699378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:30.699378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:30.699378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:30.699378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:30.699378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:30.699378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:30.699378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:30.699378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:30.699378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:30.699378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:30.699378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:30.699378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:30.700302 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:30.700302 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:30.700302 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:30.700302 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:30.702694 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 291/586] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:30.703858 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d5801f-9516-4e80-a651-7432ddbaf0e5 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:30.756251 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4889ba-286a-48cf-8029-e07a46644298 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:30.757056 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:30.792669 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997cf35-c82a-4e71-a73b-f762bce392ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:30.793506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997cf35-c82a-4e71-a73b-f762bce392ed None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:30.793799 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997cf35-c82a-4e71-a73b-f762bce392ed None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:30.794086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997cf35-c82a-4e71-a73b-f762bce392ed None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:30.802728 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997cf35-c82a-4e71-a73b-f762bce392ed None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:30.804235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997cf35-c82a-4e71-a73b-f762bce392ed None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:30.818267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d5801f-9516-4e80-a651-7432ddbaf0e5 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:30.821453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-7997cf35-c82a-4e71-a73b-f762bce392ed None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:21:30.821453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:30.821453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:30.821453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:30.821453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:30.821453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:30.821453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:30.821453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:30.821453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:30.821453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:30.821453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:30.821453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:30.821453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:30.821453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:30.821453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:30.821453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:30.822404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:30.822404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:30.822404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:30.822404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:30.822404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:30.822404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:30.822404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:30.822404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:30.822404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:30.822404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:30.822404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:30.822404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:30.822404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:30.822404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:30.822404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:30.822404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:30.822404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:30.822404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:30.823410 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:30.823410 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:30.823410 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:30.823410 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:30.823410 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:30.823410 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:30.823410 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:30.823410 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:30.823410 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:21:30.823410 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:30.823410 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 296/587] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:30.824040 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997cf35-c82a-4e71-a73b-f762bce392ed None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:30.859159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495d5bf-e161-4956-bea2-852f59195e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:30.860796 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495d5bf-e161-4956-bea2-852f59195e99 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:30.861497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495d5bf-e161-4956-bea2-852f59195e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:30.861988 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495d5bf-e161-4956-bea2-852f59195e99 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:30.863118 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0495d5bf-e161-4956-bea2-852f59195e99 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:30.863648 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0495d5bf-e161-4956-bea2-852f59195e99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:30.865420 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0495d5bf-e161-4956-bea2-852f59195e99 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:30.884632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 292/588] 10.222.0.40 () {56 vars in 1137 bytes} [Thu Jun 16 17:21:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:30.885816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495d5bf-e161-4956-bea2-852f59195e99 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:30.918988 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:30.945740 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997cf35-c82a-4e71-a73b-f762bce392ed None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:30.984731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a4902-bc51-4f69-993b-1211aecdd492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:30.985522 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a4902-bc51-4f69-993b-1211aecdd492 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:30.985871 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a4902-bc51-4f69-993b-1211aecdd492 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:30.986195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a4902-bc51-4f69-993b-1211aecdd492 None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:30.986945 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a4902-bc51-4f69-993b-1211aecdd492 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:30.987263 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a4902-bc51-4f69-993b-1211aecdd492 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:30.988924 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a4902-bc51-4f69-993b-1211aecdd492 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:30.997849 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495d5bf-e161-4956-bea2-852f59195e99 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:31.001077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:31.008243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 297/589] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:31.010333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a4902-bc51-4f69-993b-1211aecdd492 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:31.043990 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5c3c7-897e-4241-9348-10e46dd70cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:31.044682 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5c3c7-897e-4241-9348-10e46dd70cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:31.044975 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5c3c7-897e-4241-9348-10e46dd70cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:31.045290 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5c3c7-897e-4241-9348-10e46dd70cb0 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:31.046147 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5c3c7-897e-4241-9348-10e46dd70cb0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:31.046466 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5c3c7-897e-4241-9348-10e46dd70cb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:31.048465 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5c3c7-897e-4241-9348-10e46dd70cb0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:31.075140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 293/590] 10.222.0.40 () {56 vars in 1137 bytes} [Thu Jun 16 17:21:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:31.075140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5c3c7-897e-4241-9348-10e46dd70cb0 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:31.127140 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a4902-bc51-4f69-993b-1211aecdd492 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:31.163758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c49cb-7cff-4a8c-ac36-c6ade5a08faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:31.164925 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c49cb-7cff-4a8c-ac36-c6ade5a08faf None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:31.165448 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c49cb-7cff-4a8c-ac36-c6ade5a08faf None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:31.165945 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c49cb-7cff-4a8c-ac36-c6ade5a08faf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:31.169265 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:31.179055 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c49cb-7cff-4a8c-ac36-c6ade5a08faf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:31.180519 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c49cb-7cff-4a8c-ac36-c6ade5a08faf None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:31.198809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-587c49cb-7cff-4a8c-ac36-c6ade5a08faf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:31.198809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:31.198809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:31.198809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:31.198809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:31.198809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:31.198809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:31.198809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:31.198809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:31.198809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:31.198809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:31.198809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:31.198809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:31.198809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:31.198809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:31.199817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:31.199817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:31.199817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:31.199817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:31.199817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:31.199817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:31.199817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:31.199817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:31.199817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:31.199817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:31.199817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:31.199817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:31.199817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:31.199817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:31.199817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:31.199817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:31.199817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:31.200861 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:31.200861 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:31.200861 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:31.200861 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:31.205038 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 298/591] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:31.205961 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c49cb-7cff-4a8c-ac36-c6ade5a08faf None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:31.211620 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5c3c7-897e-4241-9348-10e46dd70cb0 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:31.252808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366a915-6fde-4190-9dea-b3b509eab2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:31.254117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366a915-6fde-4190-9dea-b3b509eab2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:31.254649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366a915-6fde-4190-9dea-b3b509eab2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:31.255147 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366a915-6fde-4190-9dea-b3b509eab2a9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:31.264892 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366a915-6fde-4190-9dea-b3b509eab2a9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:31.267373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366a915-6fde-4190-9dea-b3b509eab2a9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:31.285045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-1366a915-6fde-4190-9dea-b3b509eab2a9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:21:31.285045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:31.285045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:31.285045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:31.285045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:31.285045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:31.285045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:31.285045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:31.285045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:31.285045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:31.285045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:31.285045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:31.285045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:31.285045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:31.285045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:31.285045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:31.286080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:31.286080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:31.286080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:31.286080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:31.286080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:31.286080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:31.286080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:31.286080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:31.286080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:31.286080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:31.286080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:31.286080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:31.286080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:31.286080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:31.286080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:31.286080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:31.286080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:31.286080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:31.287055 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:31.287055 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:31.287055 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:31.287055 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:31.287055 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:31.287055 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:31.287055 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:31.287055 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:31.287055 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:21:31.287055 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:31.294280 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 294/592] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:21:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:31.295586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366a915-6fde-4190-9dea-b3b509eab2a9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:31.314376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c49cb-7cff-4a8c-ac36-c6ade5a08faf None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:31.323339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:31.350441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ea2c7-4a44-4aaa-948b-18c92c33e3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:31.351153 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ea2c7-4a44-4aaa-948b-18c92c33e3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:31.351438 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ea2c7-4a44-4aaa-948b-18c92c33e3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:31.351713 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ea2c7-4a44-4aaa-948b-18c92c33e3b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:31.362525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ea2c7-4a44-4aaa-948b-18c92c33e3b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:31.364036 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ea2c7-4a44-4aaa-948b-18c92c33e3b6 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:31.383070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-8b8ea2c7-4a44-4aaa-948b-18c92c33e3b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:31.383070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:31.383070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:31.383070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:31.383070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:31.383070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:31.383070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:31.383070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:31.383070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:31.383070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:31.383070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:31.383070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:31.383070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:31.383070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:31.383070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:31.385110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:31.385110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:31.385110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:31.385110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:31.385110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:31.385110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:31.385110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:31.385110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:31.385110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:31.385110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:31.385110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:31.385110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:31.385110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:31.385110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:31.385110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:31.385110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:31.385110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:31.386316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:31.386316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:31.386316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:31.386316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:31.389463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 299/593] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:31.390108 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ea2c7-4a44-4aaa-948b-18c92c33e3b6 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:31.419298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366a915-6fde-4190-9dea-b3b509eab2a9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:31.433387 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:31.462391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7234688-c67a-4d16-a52c-683fa6123ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:31.463643 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7234688-c67a-4d16-a52c-683fa6123ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:31.464174 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7234688-c67a-4d16-a52c-683fa6123ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:31.464667 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7234688-c67a-4d16-a52c-683fa6123ba7 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:31.465697 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7234688-c67a-4d16-a52c-683fa6123ba7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:31.466350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7234688-c67a-4d16-a52c-683fa6123ba7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:31.468167 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7234688-c67a-4d16-a52c-683fa6123ba7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:31.483378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 295/594] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:31.489008 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7234688-c67a-4d16-a52c-683fa6123ba7 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:31.510865 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ea2c7-4a44-4aaa-948b-18c92c33e3b6 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:31.546268 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9bb4b-3ebd-4387-80aa-a9d16b5d906a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:31.546947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9bb4b-3ebd-4387-80aa-a9d16b5d906a None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:31.547231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9bb4b-3ebd-4387-80aa-a9d16b5d906a None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:31.548096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9bb4b-3ebd-4387-80aa-a9d16b5d906a None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:31.549142 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9bb4b-3ebd-4387-80aa-a9d16b5d906a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:31.549475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9bb4b-3ebd-4387-80aa-a9d16b5d906a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:31.551005 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9bb4b-3ebd-4387-80aa-a9d16b5d906a None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:31.573021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 300/595] 10.222.0.40 () {56 vars in 1139 bytes} [Thu Jun 16 17:21:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:31.573021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9bb4b-3ebd-4387-80aa-a9d16b5d906a None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:31.605996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:31.625746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7234688-c67a-4d16-a52c-683fa6123ba7 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:31.662869 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ef52c-1478-49f7-ac76-d0320db15a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:31.663806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ef52c-1478-49f7-ac76-d0320db15a3d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:31.664098 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ef52c-1478-49f7-ac76-d0320db15a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:31.664541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ef52c-1478-49f7-ac76-d0320db15a3d None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:31.665621 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ef52c-1478-49f7-ac76-d0320db15a3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:31.666135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ef52c-1478-49f7-ac76-d0320db15a3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:31.667446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ef52c-1478-49f7-ac76-d0320db15a3d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:31.670321 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:31.676241 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9bb4b-3ebd-4387-80aa-a9d16b5d906a None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:31.681346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 296/596] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:31.682673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19c7f1-e341-4714-b6a9-25f347dcd85d None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:31.682969 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19c7f1-e341-4714-b6a9-25f347dcd85d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:31.683249 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19c7f1-e341-4714-b6a9-25f347dcd85d None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:31.688628 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 297/597] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:31.689460 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19c7f1-e341-4714-b6a9-25f347dcd85d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:31.715494 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2b146-e5b3-44fb-89da-2b889398b00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:31.716328 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2b146-e5b3-44fb-89da-2b889398b00f None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:31.716585 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2b146-e5b3-44fb-89da-2b889398b00f None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:31.717473 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2b146-e5b3-44fb-89da-2b889398b00f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:31.724914 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2b146-e5b3-44fb-89da-2b889398b00f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:31.726461 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2b146-e5b3-44fb-89da-2b889398b00f None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:31.741445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-43a2b146-e5b3-44fb-89da-2b889398b00f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:31.741445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:31.741445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:31.741445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:31.741445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:31.741445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:31.741445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:31.741445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:31.741445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:31.741445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:31.741445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:31.741445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:31.741445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:31.741445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:31.741445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:31.742406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:31.742406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:31.742406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:31.742406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:31.742406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:31.742406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:31.742406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:31.742406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:31.742406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:31.742406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:31.742406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:31.742406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:31.742406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:31.742406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:31.742406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:31.742406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:31.742406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:31.743358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:31.743358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:31.743358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:31.743358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:31.743358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:31.743358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:31.743358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:31.743358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:31.747250 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 301/598] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:31.748140 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2b146-e5b3-44fb-89da-2b889398b00f None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:31.802027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19c7f1-e341-4714-b6a9-25f347dcd85d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:31.838795 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a38f66-1209-4377-901d-1b3bcac138b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:31.839607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a38f66-1209-4377-901d-1b3bcac138b8 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:31.839894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a38f66-1209-4377-901d-1b3bcac138b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:31.840174 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a38f66-1209-4377-901d-1b3bcac138b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:31.852099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a38f66-1209-4377-901d-1b3bcac138b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:31.854031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a38f66-1209-4377-901d-1b3bcac138b8 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:31.856356 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:31.866727 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2b146-e5b3-44fb-89da-2b889398b00f None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:31.876744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-45a38f66-1209-4377-901d-1b3bcac138b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:31.876744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:31.876744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:31.876744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:31.876744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:31.876744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:31.876744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:31.876744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:31.876744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:31.876744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:31.876744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:31.876744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:31.876744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:31.876744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:31.876744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:31.878026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:31.878026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:31.878026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:31.878026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:31.878026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:31.878026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:31.878026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:31.878026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:31.878026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:31.878026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:31.878026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:31.878026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:31.878026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:31.878026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:31.878026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:31.878026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:31.878026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:31.879329 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:31.879329 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:31.879329 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:31.879329 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:31.879329 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 298/599] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:31.880097 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da3115-e861-4ca4-9d49-e20f19736577 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:31.880386 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da3115-e861-4ca4-9d49-e20f19736577 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:31.880665 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da3115-e861-4ca4-9d49-e20f19736577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:31.906296 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-80da3115-e861-4ca4-9d49-e20f19736577 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:31.908227 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-80da3115-e861-4ca4-9d49-e20f19736577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:31.920946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d6a49-ca45-4d8e-8285-632f1c55e077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:31.922293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d6a49-ca45-4d8e-8285-632f1c55e077 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:31.923235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d6a49-ca45-4d8e-8285-632f1c55e077 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:31.923672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d6a49-ca45-4d8e-8285-632f1c55e077 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:31.935179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d6a49-ca45-4d8e-8285-632f1c55e077 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:31.936930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d6a49-ca45-4d8e-8285-632f1c55e077 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:31.958053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-602d6a49-ca45-4d8e-8285-632f1c55e077 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:31.958053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:31.958053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:31.958053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:31.958053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:31.958053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:31.958053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:31.958053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:31.958053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:31.958053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:31.958053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:31.958053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:31.958053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:31.958053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:31.958053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:31.959046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:31.959046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:31.959046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:31.959046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:31.959046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:31.959046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:31.959046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:31.959046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:31.959046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:31.959046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:31.959046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:31.959046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:31.959046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:31.959046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:31.959046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:31.959046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:31.959046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:31.959975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:31.959975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:31.959975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:31.959975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:31.959975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:31.959975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:31.959975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:31.959975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:31.960420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 302/600] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:31.961330 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d6a49-ca45-4d8e-8285-632f1c55e077 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:31.982891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:31.989678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 299/601] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:31.989678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da3115-e861-4ca4-9d49-e20f19736577 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:32.076451 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d6a49-ca45-4d8e-8285-632f1c55e077 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:32.105622 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da3115-e861-4ca4-9d49-e20f19736577 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:32.114438 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff643eba-13c0-446c-930b-c504d77401db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:32.115386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff643eba-13c0-446c-930b-c504d77401db None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:32.115839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff643eba-13c0-446c-930b-c504d77401db None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:32.116298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff643eba-13c0-446c-930b-c504d77401db None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:32.117279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff643eba-13c0-446c-930b-c504d77401db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:32.117757 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff643eba-13c0-446c-930b-c504d77401db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:32.119331 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff643eba-13c0-446c-930b-c504d77401db None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:32.131075 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:32.136846 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 303/602] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:32.137768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff643eba-13c0-446c-930b-c504d77401db None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:32.144834 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb81d4-c638-4f42-b7bf-2d07c1c3c74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:32.145546 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb81d4-c638-4f42-b7bf-2d07c1c3c74e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:32.145839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb81d4-c638-4f42-b7bf-2d07c1c3c74e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:32.146122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb81d4-c638-4f42-b7bf-2d07c1c3c74e None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:32.146869 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb81d4-c638-4f42-b7bf-2d07c1c3c74e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:32.147185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb81d4-c638-4f42-b7bf-2d07c1c3c74e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:32.148569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb81d4-c638-4f42-b7bf-2d07c1c3c74e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:32.165760 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 300/603] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:32.165760 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb81d4-c638-4f42-b7bf-2d07c1c3c74e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:32.241980 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:32.275525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff643eba-13c0-446c-930b-c504d77401db None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:32.278131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb81d4-c638-4f42-b7bf-2d07c1c3c74e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:32.318782 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198777b5-aaca-4763-9a97-9d0374677215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:32.319708 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198777b5-aaca-4763-9a97-9d0374677215 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:32.319831 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198777b5-aaca-4763-9a97-9d0374677215 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:32.320060 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198777b5-aaca-4763-9a97-9d0374677215 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:32.324281 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7b1f3-ee26-40f2-8fc5-b9de6dd12cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:32.325131 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7b1f3-ee26-40f2-8fc5-b9de6dd12cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:32.325608 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7b1f3-ee26-40f2-8fc5-b9de6dd12cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:32.326038 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7b1f3-ee26-40f2-8fc5-b9de6dd12cc4 None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:32.326988 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7b1f3-ee26-40f2-8fc5-b9de6dd12cc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:32.327524 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7b1f3-ee26-40f2-8fc5-b9de6dd12cc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:32.329887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7b1f3-ee26-40f2-8fc5-b9de6dd12cc4 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:32.331599 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198777b5-aaca-4763-9a97-9d0374677215 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:32.339351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-198777b5-aaca-4763-9a97-9d0374677215 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:32.339351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:32.339351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:32.339351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:32.339351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:32.339351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:32.339351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:32.339351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:32.339351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:32.339351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:32.339351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:32.339351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:32.339351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:32.339351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:32.339351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:32.339351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:32.340492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:32.340492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:32.340492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:32.340492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:32.340492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:32.340492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:32.340492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:32.340492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:32.340492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:32.340492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:32.340492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:32.340492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:32.340492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:32.340492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:32.340492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:32.340492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:32.341409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:32.341409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:32.341409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:32.341409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:32.341409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:32.341409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:32.341409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:32.341409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:32.341409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:32.341409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:32.341409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:32.341409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:32.341409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 301/604] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:21:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:32.342149 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198777b5-aaca-4763-9a97-9d0374677215 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:32.353639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 304/605] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:32.353639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7b1f3-ee26-40f2-8fc5-b9de6dd12cc4 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:32.390113 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:32.445412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198777b5-aaca-4763-9a97-9d0374677215 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:32.454723 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7b1f3-ee26-40f2-8fc5-b9de6dd12cc4 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:32.478427 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5bd47-8ec3-405f-a2c3-dabf8f1796c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:32.479319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5bd47-8ec3-405f-a2c3-dabf8f1796c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:32.479607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5bd47-8ec3-405f-a2c3-dabf8f1796c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:32.479876 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5bd47-8ec3-405f-a2c3-dabf8f1796c4 None admin] PATH_INFO: `/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/327b347d6295422c99ce7b44eab00467/roles/0e34a6f9b89a4e29951fb189abacb34a` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:32.489366 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a163894-6956-4e04-9b09-a5e25fffd9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:32.490166 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a163894-6956-4e04-9b09-a5e25fffd9ed None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:32.490455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a163894-6956-4e04-9b09-a5e25fffd9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:32.490740 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a163894-6956-4e04-9b09-a5e25fffd9ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:32.494254 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5bd47-8ec3-405f-a2c3-dabf8f1796c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=35a535a305e740368b7a4d77d0ad22a0, user_id=327b347d6295422c99ce7b44eab00467, role_id=0e34a6f9b89a4e29951fb189abacb34a)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:32.495717 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5bd47-8ec3-405f-a2c3-dabf8f1796c4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:32.497606 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a163894-6956-4e04-9b09-a5e25fffd9ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:32.498977 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a163894-6956-4e04-9b09-a5e25fffd9ed None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:32.501478 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:32.509815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 302/606] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:21:32 2022] PUT /identity/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/327b347d6295422c99ce7b44eab00467/roles/0e34a6f9b89a4e29951fb189abacb34a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:21:32.510615 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5bd47-8ec3-405f-a2c3-dabf8f1796c4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:32.515672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-9a163894-6956-4e04-9b09-a5e25fffd9ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:32.515672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:32.515672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:32.515672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:32.515672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:32.515672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:32.515672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:32.515672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:32.515672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:32.515672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:32.515672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:32.515672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:32.515672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:32.515672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:32.515672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:32.517420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:32.517420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:32.517420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:32.517420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:32.517420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:32.517420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:32.517420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:32.517420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:32.517420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:32.517420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:32.517420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:32.517420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:32.517420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:32.517420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:32.517420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:32.517420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:32.517420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:32.523337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:32.523337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:32.523337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:32.523337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:32.523337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:32.523337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:32.523337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:32.523337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:32.523337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 305/607] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:32.523337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a163894-6956-4e04-9b09-a5e25fffd9ed None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:32.636852 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:32.647295 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5bd47-8ec3-405f-a2c3-dabf8f1796c4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:32.654323 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a163894-6956-4e04-9b09-a5e25fffd9ed None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:32.693687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388beff-95ca-4e3a-a9c7-76643a4f8df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:32.694421 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388beff-95ca-4e3a-a9c7-76643a4f8df0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:32.694709 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388beff-95ca-4e3a-a9c7-76643a4f8df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:32.694994 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388beff-95ca-4e3a-a9c7-76643a4f8df0 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:32.695756 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388beff-95ca-4e3a-a9c7-76643a4f8df0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:32.696070 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388beff-95ca-4e3a-a9c7-76643a4f8df0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:32.697533 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388beff-95ca-4e3a-a9c7-76643a4f8df0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:32.699645 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccd194-de0d-4e3b-a8be-e71b983a1a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:32.700547 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccd194-de0d-4e3b-a8be-e71b983a1a0e None admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:32.700838 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccd194-de0d-4e3b-a8be-e71b983a1a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:32.701117 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccd194-de0d-4e3b-a8be-e71b983a1a0e None admin] PATH_INFO: `/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/1fadecafbe2549acaa4cde5412638d8f/roles/0e34a6f9b89a4e29951fb189abacb34a` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:32.709285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 303/608] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 17:21:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:32.711548 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582a8dd-e0a2-40a5-bbdf-9e09cea5529b None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:32.712178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582a8dd-e0a2-40a5-bbdf-9e09cea5529b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:32.712467 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582a8dd-e0a2-40a5-bbdf-9e09cea5529b None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:32.714408 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 304/609] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:32.715042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582a8dd-e0a2-40a5-bbdf-9e09cea5529b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:32.716805 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ccd194-de0d-4e3b-a8be-e71b983a1a0e None admin] RBAC: Authorizing `identity:create_grant(project_id=35a535a305e740368b7a4d77d0ad22a0, user_id=1fadecafbe2549acaa4cde5412638d8f, role_id=0e34a6f9b89a4e29951fb189abacb34a)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:32.718299 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ccd194-de0d-4e3b-a8be-e71b983a1a0e None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:32.733079 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 306/610] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:21:32 2022] PUT /identity/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/1fadecafbe2549acaa4cde5412638d8f/roles/0e34a6f9b89a4e29951fb189abacb34a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:21:32.733988 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccd194-de0d-4e3b-a8be-e71b983a1a0e None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:32.794279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:32.860952 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccd194-de0d-4e3b-a8be-e71b983a1a0e None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:32.877024 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582a8dd-e0a2-40a5-bbdf-9e09cea5529b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:32.904698 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3291d0-83bb-4423-9666-5ee59fcd6f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:32.905536 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3291d0-83bb-4423-9666-5ee59fcd6f0d None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:32.905825 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3291d0-83bb-4423-9666-5ee59fcd6f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:32.906106 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3291d0-83bb-4423-9666-5ee59fcd6f0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:32.917632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcc140-043a-4e87-ae27-a2977534c222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:32.918314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcc140-043a-4e87-ae27-a2977534c222 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:32.918616 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcc140-043a-4e87-ae27-a2977534c222 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:32.918886 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcc140-043a-4e87-ae27-a2977534c222 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:32.918991 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3291d0-83bb-4423-9666-5ee59fcd6f0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:32.920608 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcc140-043a-4e87-ae27-a2977534c222 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:32.920608 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcc140-043a-4e87-ae27-a2977534c222 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:32.920768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3291d0-83bb-4423-9666-5ee59fcd6f0d None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:32.921475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcc140-043a-4e87-ae27-a2977534c222 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:32.939637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 305/611] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:32.940949 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0608521-ee1f-456d-971d-e386c59a00de None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:32.941267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0608521-ee1f-456d-971d-e386c59a00de None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:32.941552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0608521-ee1f-456d-971d-e386c59a00de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:32.944074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-4f3291d0-83bb-4423-9666-5ee59fcd6f0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:32.944074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:32.944074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:32.944074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:32.944074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:32.944074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:32.944074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:32.944074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:32.944074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:32.944074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:32.944074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:32.944074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:32.944074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:32.944074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:32.944074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:32.945090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:32.945090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:32.945090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:32.945090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:32.945090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:32.945090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:32.945090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:32.945090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:32.945090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:32.945090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:32.945090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:32.945090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:32.945090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:32.945090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:32.945090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:32.945090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:32.945090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:32.946506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:32.946506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:32.946506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:32.946506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:32.952711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 307/612] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:32.952711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3291d0-83bb-4423-9666-5ee59fcd6f0d None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:32.961454 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a0608521-ee1f-456d-971d-e386c59a00de None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:32.962652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a0608521-ee1f-456d-971d-e386c59a00de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:32.993554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:33.050607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 306/613] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:33.051895 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0608521-ee1f-456d-971d-e386c59a00de None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:33.080752 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:33.086113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3291d0-83bb-4423-9666-5ee59fcd6f0d None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:33.135219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43bf23-aa92-4b87-9a0f-e00314ee1348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:33.136124 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43bf23-aa92-4b87-9a0f-e00314ee1348 None admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:33.136411 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43bf23-aa92-4b87-9a0f-e00314ee1348 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:33.136705 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43bf23-aa92-4b87-9a0f-e00314ee1348 None admin] PATH_INFO: `/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/a303489493ce4cd2b62e876fbc8a7404/roles/0e34a6f9b89a4e29951fb189abacb34a` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:33.159341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43bf23-aa92-4b87-9a0f-e00314ee1348 None admin] RBAC: Authorizing `identity:create_grant(project_id=35a535a305e740368b7a4d77d0ad22a0, user_id=a303489493ce4cd2b62e876fbc8a7404, role_id=0e34a6f9b89a4e29951fb189abacb34a)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:33.160984 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43bf23-aa92-4b87-9a0f-e00314ee1348 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:33.168074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0608521-ee1f-456d-971d-e386c59a00de None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:33.183245 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 308/614] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:21:32 2022] PUT /identity/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/a303489493ce4cd2b62e876fbc8a7404/roles/0e34a6f9b89a4e29951fb189abacb34a => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:21:33.184655 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43bf23-aa92-4b87-9a0f-e00314ee1348 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:33.221108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7c6a4-d69b-4b13-abb0-e3dc9f276e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:33.221955 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7c6a4-d69b-4b13-abb0-e3dc9f276e49 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:33.222309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7c6a4-d69b-4b13-abb0-e3dc9f276e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:33.222661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7c6a4-d69b-4b13-abb0-e3dc9f276e49 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:33.223570 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7c6a4-d69b-4b13-abb0-e3dc9f276e49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:33.223958 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7c6a4-d69b-4b13-abb0-e3dc9f276e49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:33.225602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7c6a4-d69b-4b13-abb0-e3dc9f276e49 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:33.247183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 307/615] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:33.248761 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7c6a4-d69b-4b13-abb0-e3dc9f276e49 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:33.259954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:33.328184 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43bf23-aa92-4b87-9a0f-e00314ee1348 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:33.369165 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7c6a4-d69b-4b13-abb0-e3dc9f276e49 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:33.379413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f2b88-6e4c-4a0b-bbc9-b7d9f0f2df15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:33.380245 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f2b88-6e4c-4a0b-bbc9-b7d9f0f2df15 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:33.380529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f2b88-6e4c-4a0b-bbc9-b7d9f0f2df15 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:33.380804 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f2b88-6e4c-4a0b-bbc9-b7d9f0f2df15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:33.401584 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f2b88-6e4c-4a0b-bbc9-b7d9f0f2df15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:33.409789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab81ce-6968-4111-b2df-6526b382e68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:33.410379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-1f0f2b88-6e4c-4a0b-bbc9-b7d9f0f2df15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:33.410379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:33.410379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:33.410379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:33.410379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:33.410379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:33.410379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:33.410379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:33.410379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:33.410379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:33.410379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:33.410379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:33.410379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:33.410379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:33.410379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:33.410379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:33.411341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:33.411341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:33.411341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:33.411341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:33.411341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:33.411341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:33.411341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:33.411341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:33.411341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:33.411341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:33.411341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:33.411341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:33.411341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:33.411341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:33.411341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:33.411341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:33.412243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:33.412243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:33.412243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:33.412243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:33.412243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:33.412243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:33.412243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:33.412243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:33.412243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:33.412243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:33.412243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:33.412243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:33.412913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab81ce-6968-4111-b2df-6526b382e68f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:33.412913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab81ce-6968-4111-b2df-6526b382e68f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:33.412913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab81ce-6968-4111-b2df-6526b382e68f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:33.413106 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 309/616] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:21:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:33.417778 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f2b88-6e4c-4a0b-bbc9-b7d9f0f2df15 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:33.420498 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab81ce-6968-4111-b2df-6526b382e68f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:33.422081 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab81ce-6968-4111-b2df-6526b382e68f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:33.438479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-60ab81ce-6968-4111-b2df-6526b382e68f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:21:33.438479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:33.438479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:33.438479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:33.438479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:33.438479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:33.438479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:33.438479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:33.438479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:33.438479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:33.438479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:33.438479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:33.438479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:33.438479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:33.438479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:33.438479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:33.440739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:33.440739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:33.440739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:33.440739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:33.440739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:33.440739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:33.440739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:33.440739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:33.440739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:33.440739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:33.440739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:33.440739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:33.440739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:33.440739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:33.440739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:33.440739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:33.440739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:33.440739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:33.443038 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:33.443038 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:33.443038 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:33.443038 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:33.443038 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:33.443038 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:33.443038 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:33.443038 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:33.443038 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:21:33.443038 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:33.444304 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:33.449172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 308/617] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:21:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:33.449172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab81ce-6968-4111-b2df-6526b382e68f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:33.561837 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f2b88-6e4c-4a0b-bbc9-b7d9f0f2df15 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:33.573785 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:33.600114 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88811a6c-20c7-4c77-8ad7-eccdcdb31863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:33.600797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88811a6c-20c7-4c77-8ad7-eccdcdb31863 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:33.601088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88811a6c-20c7-4c77-8ad7-eccdcdb31863 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:33.601416 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88811a6c-20c7-4c77-8ad7-eccdcdb31863 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:33.602180 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88811a6c-20c7-4c77-8ad7-eccdcdb31863 None admin] RBAC: Adding query filter params (name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:33.602492 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88811a6c-20c7-4c77-8ad7-eccdcdb31863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:33.603968 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88811a6c-20c7-4c77-8ad7-eccdcdb31863 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:33.613688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 310/618] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 17:21:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:33.618531 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88811a6c-20c7-4c77-8ad7-eccdcdb31863 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:33.647206 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab81ce-6968-4111-b2df-6526b382e68f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:33.704101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483ed4e-e28c-4de0-b115-4a765192cbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:33.707624 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483ed4e-e28c-4de0-b115-4a765192cbf4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:33.707996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483ed4e-e28c-4de0-b115-4a765192cbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:33.708353 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483ed4e-e28c-4de0-b115-4a765192cbf4 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:33.709317 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483ed4e-e28c-4de0-b115-4a765192cbf4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:33.710725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483ed4e-e28c-4de0-b115-4a765192cbf4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:33.713654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483ed4e-e28c-4de0-b115-4a765192cbf4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:33.745376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:33.757027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 309/619] 10.222.0.40 () {56 vars in 1135 bytes} [Thu Jun 16 17:21:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:33.762661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483ed4e-e28c-4de0-b115-4a765192cbf4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:33.781284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88811a6c-20c7-4c77-8ad7-eccdcdb31863 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:33.819615 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb5e9b-6312-4326-af4b-8c64040ec5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:33.820876 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb5e9b-6312-4326-af4b-8c64040ec5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:33.821416 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb5e9b-6312-4326-af4b-8c64040ec5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:33.821988 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb5e9b-6312-4326-af4b-8c64040ec5c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:33.834608 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb5e9b-6312-4326-af4b-8c64040ec5c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:33.836578 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb5e9b-6312-4326-af4b-8c64040ec5c9 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:33.857195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-e7fb5e9b-6312-4326-af4b-8c64040ec5c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:33.857195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:33.857195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:33.857195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:33.857195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:33.857195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:33.857195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:33.857195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:33.857195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:33.857195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:33.857195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:33.857195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:33.857195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:33.857195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:33.857195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:33.858188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:33.858188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:33.858188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:33.858188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:33.858188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:33.858188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:33.858188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:33.858188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:33.858188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:33.858188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:33.858188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:33.858188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:33.858188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:33.858188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:33.858188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:33.858188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:33.858188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:33.859127 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:33.859127 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:33.859127 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:33.859127 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:33.862301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 311/620] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:21:33.867152 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb5e9b-6312-4326-af4b-8c64040ec5c9 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:33.985450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb5e9b-6312-4326-af4b-8c64040ec5c9 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:34.011275 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:34.016062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483ed4e-e28c-4de0-b115-4a765192cbf4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:34.025010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d05b1-56fc-493a-afcb-38240426908c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:34.026088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d05b1-56fc-493a-afcb-38240426908c None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:34.026631 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d05b1-56fc-493a-afcb-38240426908c None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:34.027115 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d05b1-56fc-493a-afcb-38240426908c None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:34.028055 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d05b1-56fc-493a-afcb-38240426908c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:34.028570 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d05b1-56fc-493a-afcb-38240426908c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:34.030213 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d05b1-56fc-493a-afcb-38240426908c None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:34.044464 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 312/621] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:34.054141 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d05b1-56fc-493a-afcb-38240426908c None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:34.057020 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:34.071129 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964aa3d0-0278-47dd-b579-894f9d3726a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:34.072269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964aa3d0-0278-47dd-b579-894f9d3726a2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:34.072725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964aa3d0-0278-47dd-b579-894f9d3726a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:34.073177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964aa3d0-0278-47dd-b579-894f9d3726a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:34.099281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964aa3d0-0278-47dd-b579-894f9d3726a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:34.102274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964aa3d0-0278-47dd-b579-894f9d3726a2 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:34.126589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-964aa3d0-0278-47dd-b579-894f9d3726a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:34.126589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:34.126589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:34.126589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:34.126589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:34.126589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:34.126589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:34.126589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:34.126589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:34.126589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:34.126589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:34.126589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:34.126589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:34.126589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:34.126589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:34.127728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:34.127728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:34.127728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:34.127728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:34.127728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:34.127728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:34.127728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:34.127728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:34.127728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:34.127728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:34.127728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:34.127728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:34.127728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:34.127728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:34.127728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:34.127728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:34.127728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:34.128859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:34.128859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:34.128859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:34.128859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:34.130809 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 310/622] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:21:34.142646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964aa3d0-0278-47dd-b579-894f9d3726a2 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:34.231129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d05b1-56fc-493a-afcb-38240426908c None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:34.271923 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c12a4e-cfa3-4e7a-99f4-618e0f0faaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:34.272795 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c12a4e-cfa3-4e7a-99f4-618e0f0faaeb None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:34.273235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c12a4e-cfa3-4e7a-99f4-618e0f0faaeb None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:34.273565 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964aa3d0-0278-47dd-b579-894f9d3726a2 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:34.273696 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c12a4e-cfa3-4e7a-99f4-618e0f0faaeb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:34.283724 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c12a4e-cfa3-4e7a-99f4-618e0f0faaeb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:34.285845 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c12a4e-cfa3-4e7a-99f4-618e0f0faaeb None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:34.307209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-58c12a4e-cfa3-4e7a-99f4-618e0f0faaeb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:21:34.307209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:34.307209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:34.307209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:34.307209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:34.307209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:34.307209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:34.307209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:34.307209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:34.307209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:34.307209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:34.307209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:34.307209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:34.307209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:34.307209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:34.307209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:34.308405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:34.308405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:34.308405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:34.308405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:34.308405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:34.308405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:34.308405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:34.308405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:34.308405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:34.308405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:34.308405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:34.308405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:34.308405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:34.308405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:34.308405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:34.308405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:34.308405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:34.308405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:34.309639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:34.309639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:34.309639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:34.309639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:34.309639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:34.309639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:34.309639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:34.309639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:34.309639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:21:34.309639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:34.317028 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 313/623] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jun 16 17:21:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:34.318732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c12a4e-cfa3-4e7a-99f4-618e0f0faaeb None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:34.323379 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f852a51-b74f-4693-9d78-004f040a9f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:34.324225 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f852a51-b74f-4693-9d78-004f040a9f3d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:34.324666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f852a51-b74f-4693-9d78-004f040a9f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:34.325214 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f852a51-b74f-4693-9d78-004f040a9f3d None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:34.326131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f852a51-b74f-4693-9d78-004f040a9f3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:34.326594 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f852a51-b74f-4693-9d78-004f040a9f3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:34.328117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f852a51-b74f-4693-9d78-004f040a9f3d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:34.342666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 311/624] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:34.348289 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f852a51-b74f-4693-9d78-004f040a9f3d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:34.381820 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:34.393658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:34.457047 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c12a4e-cfa3-4e7a-99f4-618e0f0faaeb None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:34.497491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d72cf-6209-4cd5-8c7a-45d19bdcbf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:34.498563 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d72cf-6209-4cd5-8c7a-45d19bdcbf6b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:34.499079 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d72cf-6209-4cd5-8c7a-45d19bdcbf6b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:34.499522 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f852a51-b74f-4693-9d78-004f040a9f3d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:34.499790 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d72cf-6209-4cd5-8c7a-45d19bdcbf6b None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:34.500888 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d72cf-6209-4cd5-8c7a-45d19bdcbf6b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:34.501467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d72cf-6209-4cd5-8c7a-45d19bdcbf6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:34.503315 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d72cf-6209-4cd5-8c7a-45d19bdcbf6b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:34.528611 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 314/625] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jun 16 17:21:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:34.530904 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d72cf-6209-4cd5-8c7a-45d19bdcbf6b None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:34.592271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb2902-8519-4a05-bba5-4dd1a734c76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:34.593217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb2902-8519-4a05-bba5-4dd1a734c76a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:34.593517 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb2902-8519-4a05-bba5-4dd1a734c76a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:34.593805 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb2902-8519-4a05-bba5-4dd1a734c76a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:34.613975 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb2902-8519-4a05-bba5-4dd1a734c76a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:34.615621 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb2902-8519-4a05-bba5-4dd1a734c76a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:34.650380 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d72cf-6209-4cd5-8c7a-45d19bdcbf6b None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:34.655384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:34.656878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-e2cb2902-8519-4a05-bba5-4dd1a734c76a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:34.656878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:34.656878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:34.656878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:34.656878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:34.656878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:34.656878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:34.656878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:34.656878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:34.656878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:34.656878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:34.656878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:34.656878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:34.656878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:34.656878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:34.658273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:34.658273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:34.658273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:34.658273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:34.658273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:34.658273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:34.658273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:34.658273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:34.658273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:34.658273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:34.658273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:34.658273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:34.658273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:34.658273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:34.658273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:34.658273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:34.658273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:34.659678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:34.659678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:34.659678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:34.659678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:34.659678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:34.659678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:34.659678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:34.659678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:34.659678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 312/626] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:34.664481 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb2902-8519-4a05-bba5-4dd1a734c76a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:34.697819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818f6a5-8851-4e44-9713-75cf7d4e54da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:34.699337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818f6a5-8851-4e44-9713-75cf7d4e54da None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:34.699837 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818f6a5-8851-4e44-9713-75cf7d4e54da None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:34.700341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818f6a5-8851-4e44-9713-75cf7d4e54da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:34.778521 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818f6a5-8851-4e44-9713-75cf7d4e54da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:34.780390 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818f6a5-8851-4e44-9713-75cf7d4e54da None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:34.897316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 315/627] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:21:34.900318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818f6a5-8851-4e44-9713-75cf7d4e54da None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:34.913508 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:34.974649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:34.981432 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb2902-8519-4a05-bba5-4dd1a734c76a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:35.021140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206f961-a1cf-4572-ab5b-e8098e93a251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:35.021848 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206f961-a1cf-4572-ab5b-e8098e93a251 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:35.022137 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206f961-a1cf-4572-ab5b-e8098e93a251 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:35.022420 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206f961-a1cf-4572-ab5b-e8098e93a251 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:35.023415 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206f961-a1cf-4572-ab5b-e8098e93a251 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:35.024061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206f961-a1cf-4572-ab5b-e8098e93a251 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:35.025833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206f961-a1cf-4572-ab5b-e8098e93a251 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:35.027074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818f6a5-8851-4e44-9713-75cf7d4e54da None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:35.042258 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 313/628] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:35.052510 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206f961-a1cf-4572-ab5b-e8098e93a251 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:35.098103 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b96fd-b283-439e-b5a3-081c49b9a6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:35.099284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b96fd-b283-439e-b5a3-081c49b9a6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:35.099851 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b96fd-b283-439e-b5a3-081c49b9a6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:35.100335 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b96fd-b283-439e-b5a3-081c49b9a6b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:35.127335 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b96fd-b283-439e-b5a3-081c49b9a6b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:35.129291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b96fd-b283-439e-b5a3-081c49b9a6b3 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:35.169175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206f961-a1cf-4572-ab5b-e8098e93a251 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:35.174653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-425b96fd-b283-439e-b5a3-081c49b9a6b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:35.174653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:35.174653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:35.174653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:35.174653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:35.174653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:35.174653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:35.174653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:35.174653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:35.174653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:35.174653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:35.174653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:35.174653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:35.174653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:35.174653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:35.175571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:35.175571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:35.175571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:35.175571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:35.175571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:35.175571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:35.175571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:35.175571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:35.175571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:35.175571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:35.175571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:35.175571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:35.175571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:35.175571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:35.175571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:35.175571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:35.175571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:35.176500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:35.176500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:35.176500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:35.176500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:35.182940 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 316/629] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:35.184466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b96fd-b283-439e-b5a3-081c49b9a6b3 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:35.215218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef5402-dfc0-4fc0-92ef-fc9a3f40c368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:35.216750 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef5402-dfc0-4fc0-92ef-fc9a3f40c368 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:35.217046 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef5402-dfc0-4fc0-92ef-fc9a3f40c368 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:35.217647 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef5402-dfc0-4fc0-92ef-fc9a3f40c368 None admin] PATH_INFO: `/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/b8a37d1201d54afdad9b228ef4f2d787/roles/0e34a6f9b89a4e29951fb189abacb34a` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:35.247421 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef5402-dfc0-4fc0-92ef-fc9a3f40c368 None admin] RBAC: Authorizing `identity:create_grant(project_id=35a535a305e740368b7a4d77d0ad22a0, user_id=b8a37d1201d54afdad9b228ef4f2d787, role_id=0e34a6f9b89a4e29951fb189abacb34a)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:35.249004 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef5402-dfc0-4fc0-92ef-fc9a3f40c368 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:35.258036 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:35.277187 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 314/630] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:21:35 2022] PUT /identity/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/b8a37d1201d54afdad9b228ef4f2d787/roles/0e34a6f9b89a4e29951fb189abacb34a => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:21:35.277187 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb467b2-577a-4d75-a7f6-f00310cd5ba2 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:35.277187 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb467b2-577a-4d75-a7f6-f00310cd5ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:35.277187 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb467b2-577a-4d75-a7f6-f00310cd5ba2 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:35.283698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 315/631] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:35.286232 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d3a42-f6cb-4964-900f-c8fc0ceb9418 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:35.286523 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d3a42-f6cb-4964-900f-c8fc0ceb9418 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:35.286801 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d3a42-f6cb-4964-900f-c8fc0ceb9418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:35.303253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:35.313933 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d61d3a42-f6cb-4964-900f-c8fc0ceb9418 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:35.315427 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d61d3a42-f6cb-4964-900f-c8fc0ceb9418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:35.332093 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b96fd-b283-439e-b5a3-081c49b9a6b3 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:35.373758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7f186-daf7-45d0-90f9-85cdb29cea59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:35.374552 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7f186-daf7-45d0-90f9-85cdb29cea59 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:35.374865 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7f186-daf7-45d0-90f9-85cdb29cea59 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:35.375147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7f186-daf7-45d0-90f9-85cdb29cea59 None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:35.376642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7f186-daf7-45d0-90f9-85cdb29cea59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:35.376955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7f186-daf7-45d0-90f9-85cdb29cea59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:35.378466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7f186-daf7-45d0-90f9-85cdb29cea59 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:35.394173 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 317/632] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:35.395577 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84814d9-2161-44dd-92e0-c5b0b49d073f None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:35.395874 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84814d9-2161-44dd-92e0-c5b0b49d073f None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:35.396155 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84814d9-2161-44dd-92e0-c5b0b49d073f None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:35.398049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 318/633] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:35.399242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84814d9-2161-44dd-92e0-c5b0b49d073f None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:35.413328 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 316/634] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:35.414746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ddffb-1bbc-423c-b2ee-7785eb930d1c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:35.415050 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ddffb-1bbc-423c-b2ee-7785eb930d1c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:35.415338 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ddffb-1bbc-423c-b2ee-7785eb930d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:35.435421 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-7c9ddffb-1bbc-423c-b2ee-7785eb930d1c None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:35.437618 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7c9ddffb-1bbc-423c-b2ee-7785eb930d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:35.541209 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 317/635] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:35.541209 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ddffb-1bbc-423c-b2ee-7785eb930d1c None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:35.573545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84814d9-2161-44dd-92e0-c5b0b49d073f None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:35.590297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:35.611422 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6164453-5cbe-483c-a956-e13cfcd5a8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:35.612239 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6164453-5cbe-483c-a956-e13cfcd5a8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:35.612525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6164453-5cbe-483c-a956-e13cfcd5a8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:35.612804 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6164453-5cbe-483c-a956-e13cfcd5a8a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:35.619859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6164453-5cbe-483c-a956-e13cfcd5a8a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:35.621336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6164453-5cbe-483c-a956-e13cfcd5a8a2 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:35.621874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:35.636591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-a6164453-5cbe-483c-a956-e13cfcd5a8a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:35.636591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:35.636591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:35.636591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:35.636591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:35.636591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:35.636591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:35.636591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:35.636591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:35.636591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:35.636591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:35.636591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:35.636591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:35.636591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:35.636591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:35.637576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:35.637576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:35.637576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:35.637576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:35.637576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:35.637576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:35.637576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:35.637576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:35.637576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:35.637576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:35.637576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:35.637576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:35.637576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:35.637576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:35.637576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:35.637576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:35.637576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:35.638506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:35.638506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:35.638506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:35.638506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:35.638506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:35.638506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:35.638506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:35.638506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:35.638506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 319/636] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:35.639018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6164453-5cbe-483c-a956-e13cfcd5a8a2 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:35.659138 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ddffb-1bbc-423c-b2ee-7785eb930d1c None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:35.697430 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606baaa0-336c-4621-b707-fec79cb063e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:35.698359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606baaa0-336c-4621-b707-fec79cb063e7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:35.698685 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606baaa0-336c-4621-b707-fec79cb063e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:35.699010 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606baaa0-336c-4621-b707-fec79cb063e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:35.710612 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606baaa0-336c-4621-b707-fec79cb063e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:35.719177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-606baaa0-336c-4621-b707-fec79cb063e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:35.719177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:35.719177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:35.719177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:35.719177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:35.719177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:35.719177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:35.719177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:35.719177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:35.719177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:35.719177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:35.719177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:35.719177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:35.719177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:35.719177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:35.719177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:35.720312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:35.720312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:35.720312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:35.720312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:35.720312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:35.720312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:35.720312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:35.720312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:35.720312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:35.720312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:35.720312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:35.720312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:35.720312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:35.720312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:35.720312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:35.720312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:35.721226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:35.721226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:35.721226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:35.721226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:35.721226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:35.721226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:35.721226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:35.721226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:35.721226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:35.721226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:35.721226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:35.721226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:35.725465 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 318/637] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:21:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:35.726433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606baaa0-336c-4621-b707-fec79cb063e7 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:35.743167 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6164453-5cbe-483c-a956-e13cfcd5a8a2 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:35.780072 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f862c-4784-4b53-90da-f4b80110008b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:35.780894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f862c-4784-4b53-90da-f4b80110008b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:35.781218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f862c-4784-4b53-90da-f4b80110008b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:35.781571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f862c-4784-4b53-90da-f4b80110008b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:35.793184 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764f862c-4784-4b53-90da-f4b80110008b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:35.801764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-764f862c-4784-4b53-90da-f4b80110008b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:35.801764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:35.801764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:35.801764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:35.801764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:35.801764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:35.801764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:35.801764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:35.801764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:35.801764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:35.801764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:35.801764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:35.801764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:35.801764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:35.801764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:35.801764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:35.803935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:35.803935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:35.803935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:35.803935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:35.803935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:35.803935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:35.803935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:35.803935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:35.803935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:35.803935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:35.803935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:35.803935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:35.803935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:35.803935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:35.803935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:35.803935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:35.806077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:35.806077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:35.806077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:35.806077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:35.806077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:35.806077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:35.806077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:35.806077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:35.806077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:35.806077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:35.806077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:35.806077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:35.806077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 320/638] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:21:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:35.806077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f862c-4784-4b53-90da-f4b80110008b None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:35.838105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606baaa0-336c-4621-b707-fec79cb063e7 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:35.847278 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:35.870443 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:35.880954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531a65a-3609-4590-b66b-947019902dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:35.882000 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531a65a-3609-4590-b66b-947019902dfd None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:35.882441 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531a65a-3609-4590-b66b-947019902dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:35.882876 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531a65a-3609-4590-b66b-947019902dfd None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:35.884099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531a65a-3609-4590-b66b-947019902dfd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:35.884659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531a65a-3609-4590-b66b-947019902dfd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:35.886401 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531a65a-3609-4590-b66b-947019902dfd None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:35.900862 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 319/639] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:35.902101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531a65a-3609-4590-b66b-947019902dfd None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:35.930757 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f862c-4784-4b53-90da-f4b80110008b None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:35.968618 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e49d1-1e3c-4ee6-82d0-2f296533b859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:35.970243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e49d1-1e3c-4ee6-82d0-2f296533b859 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:35.970772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e49d1-1e3c-4ee6-82d0-2f296533b859 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:35.971272 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e49d1-1e3c-4ee6-82d0-2f296533b859 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:35.972359 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e49d1-1e3c-4ee6-82d0-2f296533b859 None admin] RBAC: Adding query filter params (name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:35.972635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e49d1-1e3c-4ee6-82d0-2f296533b859 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:35.974403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e49d1-1e3c-4ee6-82d0-2f296533b859 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:35.983000 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 321/640] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 17:21:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:35.984188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e49d1-1e3c-4ee6-82d0-2f296533b859 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:36.015517 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531a65a-3609-4590-b66b-947019902dfd None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:36.052975 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36d736-5b5e-4d09-addc-49d6404f7dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:36.053706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36d736-5b5e-4d09-addc-49d6404f7dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:36.053997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36d736-5b5e-4d09-addc-49d6404f7dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:36.054285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36d736-5b5e-4d09-addc-49d6404f7dd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:36.055041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36d736-5b5e-4d09-addc-49d6404f7dd0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:36.055356 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36d736-5b5e-4d09-addc-49d6404f7dd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:36.056735 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36d736-5b5e-4d09-addc-49d6404f7dd0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:36.070827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 320/641] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 17:21:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:36.070827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bdca6-d3ab-4ea0-9748-091ff40ee6e4 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:36.070827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bdca6-d3ab-4ea0-9748-091ff40ee6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:36.070827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bdca6-d3ab-4ea0-9748-091ff40ee6e4 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:36.070827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 321/642] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:36.070827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50bdca6-d3ab-4ea0-9748-091ff40ee6e4 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:36.095771 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:36.103023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e49d1-1e3c-4ee6-82d0-2f296533b859 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:36.127114 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:36.142293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eaf11e-35d1-4d18-9484-181e7a7602a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:36.143006 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eaf11e-35d1-4d18-9484-181e7a7602a0 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:36.143291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eaf11e-35d1-4d18-9484-181e7a7602a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:36.143582 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eaf11e-35d1-4d18-9484-181e7a7602a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:36.144425 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eaf11e-35d1-4d18-9484-181e7a7602a0 None admin] RBAC: Adding query filter params (role.id=0e34a6f9b89a4e29951fb189abacb34a, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=689a31e26c6b454a91c3fd79678b4d07) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:36.144750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eaf11e-35d1-4d18-9484-181e7a7602a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:36.146228 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eaf11e-35d1-4d18-9484-181e7a7602a0 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:36.161182 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 322/643] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:21:35 2022] GET /identity/v3/role_assignments?user.id=689a31e26c6b454a91c3fd79678b4d07&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=0e34a6f9b89a4e29951fb189abacb34a => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:36.161182 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eaf11e-35d1-4d18-9484-181e7a7602a0 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:36.193205 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50bdca6-d3ab-4ea0-9748-091ff40ee6e4 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:36.243256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349657c5-1fc6-489c-9ba3-7a8388b94c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:36.244077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349657c5-1fc6-489c-9ba3-7a8388b94c48 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:36.244367 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349657c5-1fc6-489c-9ba3-7a8388b94c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:36.244642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349657c5-1fc6-489c-9ba3-7a8388b94c48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:36.259489 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349657c5-1fc6-489c-9ba3-7a8388b94c48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:36.261098 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349657c5-1fc6-489c-9ba3-7a8388b94c48 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:36.281412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-349657c5-1fc6-489c-9ba3-7a8388b94c48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:36.281412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:36.281412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:36.281412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:36.281412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:36.281412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:36.281412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:36.281412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:36.281412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:36.281412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:36.281412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:36.281412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:36.281412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:36.281412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:36.281412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:36.282924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:36.282924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:36.282924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:36.282924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:36.282924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:36.282924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:36.282924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:36.282924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:36.282924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:36.282924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:36.282924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:36.282924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:36.282924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:36.282924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:36.282924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:36.282924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:36.282924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:36.284090 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:36.284090 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:36.284090 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:36.284090 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:36.287379 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 322/644] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:36.288693 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a170f6-3a3b-492a-8621-21a62ff15f82 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:36.288993 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a170f6-3a3b-492a-8621-21a62ff15f82 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:36.289300 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a170f6-3a3b-492a-8621-21a62ff15f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:36.307554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-41a170f6-3a3b-492a-8621-21a62ff15f82 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:36.308774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-41a170f6-3a3b-492a-8621-21a62ff15f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:36.315616 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eaf11e-35d1-4d18-9484-181e7a7602a0 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:36.369967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a24e0-1d4f-47d1-ae81-486222d3550e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:36.371123 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a24e0-1d4f-47d1-ae81-486222d3550e None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:36.371650 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a24e0-1d4f-47d1-ae81-486222d3550e None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:36.371938 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a24e0-1d4f-47d1-ae81-486222d3550e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:36.387348 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a24e0-1d4f-47d1-ae81-486222d3550e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:36.389021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a24e0-1d4f-47d1-ae81-486222d3550e None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:36.396396 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 323/645] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:36.397535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a170f6-3a3b-492a-8621-21a62ff15f82 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:36.411743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-aa5a24e0-1d4f-47d1-ae81-486222d3550e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:36.411743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:36.411743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:36.411743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:36.411743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:36.411743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:36.411743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:36.411743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:36.411743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:36.411743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:36.411743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:36.411743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:36.411743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:36.411743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:36.411743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:36.412717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:36.412717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:36.412717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:36.412717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:36.412717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:36.412717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:36.412717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:36.412717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:36.412717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:36.412717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:36.412717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:36.412717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:36.412717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:36.412717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:36.412717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:36.412717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:36.412717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:36.413883 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:36.413883 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:36.413883 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:36.413883 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:36.413883 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 323/646] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:36.419551 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a24e0-1d4f-47d1-ae81-486222d3550e None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:36.433821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:36.446269 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:36.543773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a170f6-3a3b-492a-8621-21a62ff15f82 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:36.559402 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a24e0-1d4f-47d1-ae81-486222d3550e None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:36.583851 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ddcc89-7ab4-48ac-94c0-fa9de6c3fb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:36.584984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddcc89-7ab4-48ac-94c0-fa9de6c3fb6a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:36.585621 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddcc89-7ab4-48ac-94c0-fa9de6c3fb6a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:36.586022 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddcc89-7ab4-48ac-94c0-fa9de6c3fb6a None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:36.586835 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ddcc89-7ab4-48ac-94c0-fa9de6c3fb6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:36.587158 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ddcc89-7ab4-48ac-94c0-fa9de6c3fb6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:36.589099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ddcc89-7ab4-48ac-94c0-fa9de6c3fb6a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:36.606929 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 324/647] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:36.606929 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ddcc89-7ab4-48ac-94c0-fa9de6c3fb6a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:36.618251 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e7182-f765-41af-823a-40e8aa34e612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:36.619942 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e7182-f765-41af-823a-40e8aa34e612 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:36.620557 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e7182-f765-41af-823a-40e8aa34e612 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:36.621050 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e7182-f765-41af-823a-40e8aa34e612 None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:36.622492 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e7182-f765-41af-823a-40e8aa34e612 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:36.623419 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e7182-f765-41af-823a-40e8aa34e612 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:36.625508 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e7182-f765-41af-823a-40e8aa34e612 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:36.645067 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 324/648] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:36.646348 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e7182-f765-41af-823a-40e8aa34e612 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:36.713134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:36.725990 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ddcc89-7ab4-48ac-94c0-fa9de6c3fb6a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:36.745270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:36.763026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c6d88-12c3-4d89-84fb-fa3ce1feb862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:36.763873 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c6d88-12c3-4d89-84fb-fa3ce1feb862 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:36.764161 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c6d88-12c3-4d89-84fb-fa3ce1feb862 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:36.764441 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c6d88-12c3-4d89-84fb-fa3ce1feb862 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:36.775988 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c6d88-12c3-4d89-84fb-fa3ce1feb862 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:36.784557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-f65c6d88-12c3-4d89-84fb-fa3ce1feb862 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:36.784557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:36.784557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:36.784557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:36.784557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:36.784557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:36.784557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:36.784557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:36.784557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:36.784557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:36.784557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:36.784557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:36.784557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:36.784557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:36.784557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:36.784557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:36.785789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:36.785789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:36.785789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:36.785789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:36.785789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:36.785789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:36.785789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:36.785789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:36.785789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:36.785789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:36.785789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:36.785789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:36.785789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:36.785789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:36.785789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:36.785789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:36.786670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:36.786670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:36.786670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:36.786670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:36.786670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:36.786670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:36.786670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:36.786670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:36.786670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:36.786670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:36.786670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:36.786670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:36.790528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 325/649] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:21:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:36.791358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c6d88-12c3-4d89-84fb-fa3ce1feb862 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:36.792183 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e7182-f765-41af-823a-40e8aa34e612 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:36.839425 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0896c10-81b4-4e29-a0e9-0d61f1e7a147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:36.841273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0896c10-81b4-4e29-a0e9-0d61f1e7a147 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:36.842243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0896c10-81b4-4e29-a0e9-0d61f1e7a147 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:36.843218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0896c10-81b4-4e29-a0e9-0d61f1e7a147 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:36.853769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0896c10-81b4-4e29-a0e9-0d61f1e7a147 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:36.856084 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0896c10-81b4-4e29-a0e9-0d61f1e7a147 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:36.875894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-a0896c10-81b4-4e29-a0e9-0d61f1e7a147 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:21:36.875894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:36.875894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:36.875894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:36.875894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:36.875894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:36.875894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:36.875894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:36.875894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:36.875894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:36.875894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:36.875894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:36.875894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:36.875894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:36.875894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:36.875894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:36.876918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:36.876918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:36.876918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:36.876918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:36.876918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:36.876918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:36.876918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:36.876918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:36.876918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:36.876918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:36.876918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:36.876918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:36.876918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:36.876918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:36.876918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:36.876918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:36.876918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:36.876918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:36.877947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:36.877947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:36.877947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:36.877947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:36.877947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:36.877947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:36.877947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:36.877947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:36.877947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:21:36.877947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:36.884014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 325/650] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:36.884014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0896c10-81b4-4e29-a0e9-0d61f1e7a147 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:36.913906 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c6d88-12c3-4d89-84fb-fa3ce1feb862 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:36.961884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdffec1-8f8b-47a3-bd25-2231b5411f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:36.962787 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdffec1-8f8b-47a3-bd25-2231b5411f63 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:36.963143 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdffec1-8f8b-47a3-bd25-2231b5411f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:36.963491 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdffec1-8f8b-47a3-bd25-2231b5411f63 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:36.971773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdffec1-8f8b-47a3-bd25-2231b5411f63 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:36.973376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdffec1-8f8b-47a3-bd25-2231b5411f63 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:36.988912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-9bdffec1-8f8b-47a3-bd25-2231b5411f63 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:21:36.988912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:36.988912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:36.988912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:36.988912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:36.988912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:36.988912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:36.988912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:36.988912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:36.988912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:36.988912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:36.988912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:36.988912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:36.988912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:36.988912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:36.988912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:36.989973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:36.989973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:36.989973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:36.989973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:36.989973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:36.989973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:36.989973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:36.989973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:36.989973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:36.989973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:36.989973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:36.989973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:36.989973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:36.989973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:36.989973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:36.989973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:36.989973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:36.989973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:36.990946 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:36.990946 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:36.990946 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:36.990946 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:36.990946 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:36.990946 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:36.990946 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:36.990946 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:36.990946 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:21:36.990946 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:36.994992 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 326/651] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:36.995933 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdffec1-8f8b-47a3-bd25-2231b5411f63 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:37.002730 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0896c10-81b4-4e29-a0e9-0d61f1e7a147 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:37.010144 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:37.042894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:37.051079 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458a9a9-b89a-4d9c-9a9c-507cc6731c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:37.052048 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458a9a9-b89a-4d9c-9a9c-507cc6731c24 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:37.052690 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458a9a9-b89a-4d9c-9a9c-507cc6731c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:37.053196 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458a9a9-b89a-4d9c-9a9c-507cc6731c24 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:37.054397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458a9a9-b89a-4d9c-9a9c-507cc6731c24 None admin] RBAC: Adding query filter params (name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:37.054879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458a9a9-b89a-4d9c-9a9c-507cc6731c24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:37.056422 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458a9a9-b89a-4d9c-9a9c-507cc6731c24 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:37.072074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 326/652] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 17:21:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:37.073064 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458a9a9-b89a-4d9c-9a9c-507cc6731c24 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:37.119457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdffec1-8f8b-47a3-bd25-2231b5411f63 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:37.165658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5a5cd-d022-4683-8d9b-90cc7fba6426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:37.166849 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5a5cd-d022-4683-8d9b-90cc7fba6426 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:37.167317 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5a5cd-d022-4683-8d9b-90cc7fba6426 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:37.167887 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5a5cd-d022-4683-8d9b-90cc7fba6426 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:37.169294 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f5a5cd-d022-4683-8d9b-90cc7fba6426 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:37.169768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f5a5cd-d022-4683-8d9b-90cc7fba6426 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:37.171795 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f5a5cd-d022-4683-8d9b-90cc7fba6426 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:37.191670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458a9a9-b89a-4d9c-9a9c-507cc6731c24 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:37.200899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 327/653] 10.222.0.40 () {56 vars in 1137 bytes} [Thu Jun 16 17:21:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:37.202005 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5a5cd-d022-4683-8d9b-90cc7fba6426 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:37.243013 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39b8de-e832-4a3c-b654-4e6846376ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:37.243635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39b8de-e832-4a3c-b654-4e6846376ded None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:37.244023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39b8de-e832-4a3c-b654-4e6846376ded None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:37.244429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39b8de-e832-4a3c-b654-4e6846376ded None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:37.245396 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39b8de-e832-4a3c-b654-4e6846376ded None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:37.245804 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39b8de-e832-4a3c-b654-4e6846376ded None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:37.247378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39b8de-e832-4a3c-b654-4e6846376ded None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:37.273555 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 327/654] 10.222.0.40 () {56 vars in 1137 bytes} [Thu Jun 16 17:21:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:37.274558 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39b8de-e832-4a3c-b654-4e6846376ded None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:37.296375 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:37.335568 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5a5cd-d022-4683-8d9b-90cc7fba6426 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:37.343736 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:37.373464 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592e04a-189e-460c-a3d7-eedbb86e40dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:37.374649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592e04a-189e-460c-a3d7-eedbb86e40dd None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:37.375218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592e04a-189e-460c-a3d7-eedbb86e40dd None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:37.375786 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592e04a-189e-460c-a3d7-eedbb86e40dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:37.392198 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f592e04a-189e-460c-a3d7-eedbb86e40dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:37.394100 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f592e04a-189e-460c-a3d7-eedbb86e40dd None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:37.405447 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39b8de-e832-4a3c-b654-4e6846376ded None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:37.417545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-f592e04a-189e-460c-a3d7-eedbb86e40dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:37.417545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:37.417545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:37.417545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:37.417545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:37.417545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:37.417545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:37.417545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:37.417545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:37.417545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:37.417545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:37.417545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:37.417545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:37.417545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:37.417545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:37.418476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:37.418476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:37.418476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:37.418476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:37.418476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:37.418476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:37.418476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:37.418476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:37.418476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:37.418476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:37.418476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:37.418476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:37.418476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:37.418476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:37.418476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:37.418476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:37.418476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:37.419396 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:37.419396 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:37.419396 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:37.419396 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:37.423285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 328/655] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:37.424187 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592e04a-189e-460c-a3d7-eedbb86e40dd None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:37.452936 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8a7d3-fc3d-4503-96a1-342b987b7b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:37.454811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8a7d3-fc3d-4503-96a1-342b987b7b7e None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:37.455503 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8a7d3-fc3d-4503-96a1-342b987b7b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:37.456043 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8a7d3-fc3d-4503-96a1-342b987b7b7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:37.540973 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8a7d3-fc3d-4503-96a1-342b987b7b7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:37.542477 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8a7d3-fc3d-4503-96a1-342b987b7b7e None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:37.554576 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592e04a-189e-460c-a3d7-eedbb86e40dd None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:37.579326 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:37.597658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad6094-46e5-444b-a27e-c002a889489b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:37.598629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad6094-46e5-444b-a27e-c002a889489b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:37.598911 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad6094-46e5-444b-a27e-c002a889489b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:37.599116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:37.599232 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad6094-46e5-444b-a27e-c002a889489b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:37.655034 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 328/656] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:21:37.655034 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8a7d3-fc3d-4503-96a1-342b987b7b7e None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:37.677100 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad6094-46e5-444b-a27e-c002a889489b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:37.678596 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad6094-46e5-444b-a27e-c002a889489b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:37.761589 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8a7d3-fc3d-4503-96a1-342b987b7b7e None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:37.787327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 329/657] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:21:37.788674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef5c3c-fb4a-49b7-82a1-6bf9f5120551 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:37.788967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef5c3c-fb4a-49b7-82a1-6bf9f5120551 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:37.789273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef5c3c-fb4a-49b7-82a1-6bf9f5120551 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:37.790703 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 330/658] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:37.794238 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef5c3c-fb4a-49b7-82a1-6bf9f5120551 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:37.803091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410d6cb-7ba0-4b54-a62e-ed4097b2026a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:37.803759 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410d6cb-7ba0-4b54-a62e-ed4097b2026a None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:37.804047 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410d6cb-7ba0-4b54-a62e-ed4097b2026a None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:37.804443 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410d6cb-7ba0-4b54-a62e-ed4097b2026a None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:37.805190 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0410d6cb-7ba0-4b54-a62e-ed4097b2026a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:37.805506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0410d6cb-7ba0-4b54-a62e-ed4097b2026a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:37.807006 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0410d6cb-7ba0-4b54-a62e-ed4097b2026a None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:37.819515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:37.821939 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 329/659] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:37.822839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410d6cb-7ba0-4b54-a62e-ed4097b2026a None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:37.839545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:37.917323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef5c3c-fb4a-49b7-82a1-6bf9f5120551 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:37.935551 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410d6cb-7ba0-4b54-a62e-ed4097b2026a None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:37.952088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa6324-cbda-4b08-8408-5536116c87ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:37.952896 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa6324-cbda-4b08-8408-5536116c87ea None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:37.953207 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa6324-cbda-4b08-8408-5536116c87ea None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:37.954039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa6324-cbda-4b08-8408-5536116c87ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:37.964390 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa6324-cbda-4b08-8408-5536116c87ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:37.965881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa6324-cbda-4b08-8408-5536116c87ea None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:37.973856 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db565b-78fd-4b0f-98c5-2a7768bf1cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:37.974839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db565b-78fd-4b0f-98c5-2a7768bf1cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:37.975354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db565b-78fd-4b0f-98c5-2a7768bf1cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:37.975781 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db565b-78fd-4b0f-98c5-2a7768bf1cf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:37.984166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-a6fa6324-cbda-4b08-8408-5536116c87ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:37.984166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:37.984166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:37.984166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:37.984166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:37.984166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:37.984166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:37.984166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:37.984166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:37.984166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:37.984166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:37.984166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:37.984166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:37.984166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:37.984166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:37.985274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:37.985274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:37.985274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:37.985274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:37.985274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:37.985274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:37.985274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:37.985274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:37.985274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:37.985274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:37.985274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:37.985274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:37.985274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:37.985274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:37.985274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:37.985274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:37.985274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:37.986453 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:37.986453 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:37.986453 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:37.986453 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:37.989389 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db565b-78fd-4b0f-98c5-2a7768bf1cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:37.989936 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 331/660] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:37.991255 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e6ec1-9926-480f-b779-619e0ca1c12a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:37.991445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db565b-78fd-4b0f-98c5-2a7768bf1cf3 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:37.991553 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e6ec1-9926-480f-b779-619e0ca1c12a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:37.991820 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e6ec1-9926-480f-b779-619e0ca1c12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:38.008430 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-247e6ec1-9926-480f-b779-619e0ca1c12a None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:38.009898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-247e6ec1-9926-480f-b779-619e0ca1c12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:38.010587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-61db565b-78fd-4b0f-98c5-2a7768bf1cf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:38.010587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:38.010587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:38.010587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:38.010587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:38.010587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:38.010587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:38.010587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:38.010587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:38.010587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:38.010587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:38.010587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:38.010587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:38.010587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:38.010587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:38.011668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:38.011668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:38.011668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:38.011668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:38.011668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:38.011668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:38.011668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:38.011668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:38.011668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:38.011668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:38.011668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:38.011668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:38.011668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:38.011668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:38.011668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:38.011668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:38.011668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:38.012782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:38.012782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:38.012782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:38.012782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:38.014764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 330/661] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:38.016311 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db565b-78fd-4b0f-98c5-2a7768bf1cf3 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:38.081926 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 332/662] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:38.082505 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:38.083126 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e6ec1-9926-480f-b779-619e0ca1c12a None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:38.085837 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:38.137967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db565b-78fd-4b0f-98c5-2a7768bf1cf3 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:38.174826 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4aa38-f32c-4524-981c-f226e1f393ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:38.175637 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4aa38-f32c-4524-981c-f226e1f393ad None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:38.175967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4aa38-f32c-4524-981c-f226e1f393ad None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:38.176247 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4aa38-f32c-4524-981c-f226e1f393ad None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:38.184061 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4aa38-f32c-4524-981c-f226e1f393ad None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:38.185523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4aa38-f32c-4524-981c-f226e1f393ad None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:38.192767 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e6ec1-9926-480f-b779-619e0ca1c12a None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:38.200433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-e0e4aa38-f32c-4524-981c-f226e1f393ad None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:21:38.200433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:38.200433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:38.200433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:38.200433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:38.200433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:38.200433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:38.200433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:38.200433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:38.200433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:38.200433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:38.200433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:38.200433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:38.200433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:38.200433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:38.200433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:38.201636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:38.201636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:38.201636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:38.201636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:38.201636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:38.201636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:38.201636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:38.201636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:38.201636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:38.201636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:38.201636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:38.201636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:38.201636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:38.201636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:38.201636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:38.201636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:38.201636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:38.201636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:38.202614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:38.202614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:38.202614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:38.202614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:38.202614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:38.202614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:38.202614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:38.202614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:38.202614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:21:38.202614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:38.202614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 331/663] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:21:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:38.202614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4aa38-f32c-4524-981c-f226e1f393ad None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:38.242082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f5bea-6c2f-46e5-871e-6afab1467513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:38.242773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f5bea-6c2f-46e5-871e-6afab1467513 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:38.243059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f5bea-6c2f-46e5-871e-6afab1467513 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:38.243339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f5bea-6c2f-46e5-871e-6afab1467513 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:38.244072 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f5bea-6c2f-46e5-871e-6afab1467513 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:38.244380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f5bea-6c2f-46e5-871e-6afab1467513 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:38.245776 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f5bea-6c2f-46e5-871e-6afab1467513 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:38.262148 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 333/664] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:38.262148 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f5bea-6c2f-46e5-871e-6afab1467513 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:38.319899 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4aa38-f32c-4524-981c-f226e1f393ad None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:38.328743 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:38.345261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:38.356939 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff5043-28f3-4156-809b-9592e3b26a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:38.357634 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff5043-28f3-4156-809b-9592e3b26a55 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:38.357924 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff5043-28f3-4156-809b-9592e3b26a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:38.358206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff5043-28f3-4156-809b-9592e3b26a55 None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:38.358932 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff5043-28f3-4156-809b-9592e3b26a55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:38.359241 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff5043-28f3-4156-809b-9592e3b26a55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:38.360833 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff5043-28f3-4156-809b-9592e3b26a55 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:38.373052 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 332/665] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:38.373711 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f5bea-6c2f-46e5-871e-6afab1467513 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:38.378688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff5043-28f3-4156-809b-9592e3b26a55 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:38.412097 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b441ba3d-93c4-4a3a-a61f-a2f0ef7f6c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:38.412911 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441ba3d-93c4-4a3a-a61f-a2f0ef7f6c63 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:38.413246 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441ba3d-93c4-4a3a-a61f-a2f0ef7f6c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:38.413533 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441ba3d-93c4-4a3a-a61f-a2f0ef7f6c63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:38.425525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b441ba3d-93c4-4a3a-a61f-a2f0ef7f6c63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:38.433541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-b441ba3d-93c4-4a3a-a61f-a2f0ef7f6c63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:38.433541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:38.433541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:38.433541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:38.433541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:38.433541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:38.433541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:38.433541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:38.433541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:38.433541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:38.433541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:38.433541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:38.433541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:38.433541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:38.433541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:38.433541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:38.434644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:38.434644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:38.434644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:38.434644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:38.434644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:38.434644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:38.434644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:38.434644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:38.434644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:38.434644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:38.434644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:38.434644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:38.434644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:38.434644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:38.434644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:38.434644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:38.435503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:38.435503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:38.435503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:38.435503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:38.435503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:38.435503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:38.435503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:38.435503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:38.435503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:38.435503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:38.435503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:38.435503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:38.440562 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 334/666] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:21:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:38.441366 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b441ba3d-93c4-4a3a-a61f-a2f0ef7f6c63 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:38.489685 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff5043-28f3-4156-809b-9592e3b26a55 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:38.524587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba4506-8b94-49eb-9197-69cd037c3992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:38.525282 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba4506-8b94-49eb-9197-69cd037c3992 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:38.525569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba4506-8b94-49eb-9197-69cd037c3992 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:38.525958 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba4506-8b94-49eb-9197-69cd037c3992 None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:38.526801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba4506-8b94-49eb-9197-69cd037c3992 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:38.527115 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba4506-8b94-49eb-9197-69cd037c3992 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:38.528542 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba4506-8b94-49eb-9197-69cd037c3992 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:38.545779 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b441ba3d-93c4-4a3a-a61f-a2f0ef7f6c63 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:38.546104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 333/667] 10.222.0.40 () {56 vars in 1139 bytes} [Thu Jun 16 17:21:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:38.547204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba4506-8b94-49eb-9197-69cd037c3992 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:38.572059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:38.584543 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c37834-85ab-471e-ba7a-3eb91ca652fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:38.585372 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c37834-85ab-471e-ba7a-3eb91ca652fd None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:38.585661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c37834-85ab-471e-ba7a-3eb91ca652fd None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:38.585941 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c37834-85ab-471e-ba7a-3eb91ca652fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:38.590326 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:38.592651 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c37834-85ab-471e-ba7a-3eb91ca652fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:38.594170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c37834-85ab-471e-ba7a-3eb91ca652fd None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:38.609952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-f5c37834-85ab-471e-ba7a-3eb91ca652fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:38.609952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:38.609952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:38.609952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:38.609952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:38.609952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:38.609952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:38.609952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:38.609952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:38.609952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:38.609952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:38.609952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:38.609952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:38.609952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:38.609952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:38.611053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:38.611053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:38.611053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:38.611053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:38.611053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:38.611053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:38.611053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:38.611053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:38.611053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:38.611053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:38.611053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:38.611053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:38.611053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:38.611053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:38.611053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:38.611053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:38.611053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:38.612225 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:38.612225 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:38.612225 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:38.612225 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:38.612225 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:38.612225 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:38.612225 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:38.612225 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:38.612934 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 335/668] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:38.613792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c37834-85ab-471e-ba7a-3eb91ca652fd None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:38.650777 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba4506-8b94-49eb-9197-69cd037c3992 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:38.684145 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef568c88-496c-446e-8830-c34b274895b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:38.684950 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef568c88-496c-446e-8830-c34b274895b9 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:38.685259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef568c88-496c-446e-8830-c34b274895b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:38.685541 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef568c88-496c-446e-8830-c34b274895b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:38.694394 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef568c88-496c-446e-8830-c34b274895b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:38.695857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef568c88-496c-446e-8830-c34b274895b9 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:38.709397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-ef568c88-496c-446e-8830-c34b274895b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:38.709397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:38.709397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:38.709397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:38.709397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:38.709397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:38.709397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:38.709397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:38.709397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:38.709397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:38.709397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:38.709397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:38.709397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:38.709397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:38.709397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:38.710440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:38.710440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:38.710440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:38.710440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:38.710440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:38.710440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:38.710440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:38.710440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:38.710440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:38.710440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:38.710440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:38.710440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:38.710440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:38.710440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:38.710440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:38.710440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:38.710440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:38.711378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:38.711378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:38.711378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:38.711378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:38.711378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:38.711378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:38.711378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:38.711378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:38.711378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 334/669] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:38.711876 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb72f14-7b66-4245-b54e-1cbac892e95a None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:38.712152 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb72f14-7b66-4245-b54e-1cbac892e95a None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:38.712377 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb72f14-7b66-4245-b54e-1cbac892e95a None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:38.713712 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 335/670] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:38.714394 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb72f14-7b66-4245-b54e-1cbac892e95a None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:38.720432 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c37834-85ab-471e-ba7a-3eb91ca652fd None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:38.756129 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e8c73-0f50-4d4c-859c-e4d42482ce5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:38.756756 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e8c73-0f50-4d4c-859c-e4d42482ce5d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:38.757046 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e8c73-0f50-4d4c-859c-e4d42482ce5d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:38.757356 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e8c73-0f50-4d4c-859c-e4d42482ce5d None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:38.758107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e8c73-0f50-4d4c-859c-e4d42482ce5d None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:38.758673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e8c73-0f50-4d4c-859c-e4d42482ce5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:38.760057 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e8c73-0f50-4d4c-859c-e4d42482ce5d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:38.768264 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 336/671] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 17:21:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:38.769058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e8c73-0f50-4d4c-859c-e4d42482ce5d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:38.813092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:38.825053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb72f14-7b66-4245-b54e-1cbac892e95a None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:38.827616 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:38.860509 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e361487-1a54-4c21-9c1f-bf59fe6f9e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:38.861185 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e361487-1a54-4c21-9c1f-bf59fe6f9e2c None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:38.861481 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e361487-1a54-4c21-9c1f-bf59fe6f9e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:38.861766 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e361487-1a54-4c21-9c1f-bf59fe6f9e2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:38.867985 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e8c73-0f50-4d4c-859c-e4d42482ce5d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:38.901029 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86cd22-c816-42e5-ba06-c8ab0ae5e5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:38.901792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86cd22-c816-42e5-ba06-c8ab0ae5e5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:38.902079 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86cd22-c816-42e5-ba06-c8ab0ae5e5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:38.902358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86cd22-c816-42e5-ba06-c8ab0ae5e5e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:38.903146 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce86cd22-c816-42e5-ba06-c8ab0ae5e5e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:38.903456 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce86cd22-c816-42e5-ba06-c8ab0ae5e5e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:38.904882 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce86cd22-c816-42e5-ba06-c8ab0ae5e5e2 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:38.917707 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 337/672] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:38.918493 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86cd22-c816-42e5-ba06-c8ab0ae5e5e2 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:38.933701 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e361487-1a54-4c21-9c1f-bf59fe6f9e2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:38.935157 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e361487-1a54-4c21-9c1f-bf59fe6f9e2c None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:39.018585 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86cd22-c816-42e5-ba06-c8ab0ae5e5e2 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:39.020795 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:39.033409 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 336/673] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:21:39.034740 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6209438-3820-4797-8c92-4197956decd6 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:39.035030 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6209438-3820-4797-8c92-4197956decd6 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:39.035316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6209438-3820-4797-8c92-4197956decd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:39.040271 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:39.053381 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-d6209438-3820-4797-8c92-4197956decd6 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:39.054581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-d6209438-3820-4797-8c92-4197956decd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:39.060115 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f759c08-7c09-41c1-8abe-80fb88345b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:39.060779 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f759c08-7c09-41c1-8abe-80fb88345b69 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:39.061064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f759c08-7c09-41c1-8abe-80fb88345b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:39.061368 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f759c08-7c09-41c1-8abe-80fb88345b69 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:39.062160 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f759c08-7c09-41c1-8abe-80fb88345b69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:39.062469 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f759c08-7c09-41c1-8abe-80fb88345b69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:39.064253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f759c08-7c09-41c1-8abe-80fb88345b69 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:39.075929 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 338/674] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:39.076712 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f759c08-7c09-41c1-8abe-80fb88345b69 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:39.121928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 337/675] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:39.122873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6209438-3820-4797-8c92-4197956decd6 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:39.178482 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f759c08-7c09-41c1-8abe-80fb88345b69 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:39.212253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2225dbe-f092-4c70-999d-5233d611aee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:39.213059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2225dbe-f092-4c70-999d-5233d611aee0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:39.213395 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2225dbe-f092-4c70-999d-5233d611aee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:39.213679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2225dbe-f092-4c70-999d-5233d611aee0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:39.224481 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2225dbe-f092-4c70-999d-5233d611aee0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:39.225970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2225dbe-f092-4c70-999d-5233d611aee0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:39.229201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6209438-3820-4797-8c92-4197956decd6 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:39.243298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-a2225dbe-f092-4c70-999d-5233d611aee0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:39.243298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:39.243298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:39.243298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:39.243298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:39.243298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:39.243298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:39.243298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:39.243298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:39.243298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:39.243298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:39.243298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:39.243298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:39.243298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:39.243298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:39.244279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:39.244279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:39.244279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:39.244279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:39.244279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:39.244279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:39.244279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:39.244279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:39.244279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:39.244279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:39.244279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:39.244279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:39.244279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:39.244279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:39.244279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:39.244279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:39.244279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:39.245282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:39.245282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:39.245282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:39.245282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:39.245282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 339/676] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:39.245931 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2225dbe-f092-4c70-999d-5233d611aee0 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:39.264492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:39.269907 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:39.275624 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673864ae-d2c0-458d-a0ae-d061b703ec70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:39.276289 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673864ae-d2c0-458d-a0ae-d061b703ec70 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:39.276588 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673864ae-d2c0-458d-a0ae-d061b703ec70 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:39.276869 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673864ae-d2c0-458d-a0ae-d061b703ec70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:39.277758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673864ae-d2c0-458d-a0ae-d061b703ec70 None admin] RBAC: Adding query filter params (role.id=0e34a6f9b89a4e29951fb189abacb34a, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=327b347d6295422c99ce7b44eab00467) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:39.278080 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673864ae-d2c0-458d-a0ae-d061b703ec70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:39.279513 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673864ae-d2c0-458d-a0ae-d061b703ec70 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:39.285488 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 338/677] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:21:39 2022] GET /identity/v3/role_assignments?user.id=327b347d6295422c99ce7b44eab00467&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=0e34a6f9b89a4e29951fb189abacb34a => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:39.286273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673864ae-d2c0-458d-a0ae-d061b703ec70 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:39.372704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2225dbe-f092-4c70-999d-5233d611aee0 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:39.414190 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49fbb3-7c03-4f58-a5d1-77a541903f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:39.415021 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49fbb3-7c03-4f58-a5d1-77a541903f47 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:39.415312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49fbb3-7c03-4f58-a5d1-77a541903f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:39.415592 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49fbb3-7c03-4f58-a5d1-77a541903f47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:39.415692 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673864ae-d2c0-458d-a0ae-d061b703ec70 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:39.428096 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49fbb3-7c03-4f58-a5d1-77a541903f47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:39.429621 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49fbb3-7c03-4f58-a5d1-77a541903f47 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:39.448632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-8d49fbb3-7c03-4f58-a5d1-77a541903f47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:39.448632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:39.448632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:39.448632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:39.448632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:39.448632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:39.448632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:39.448632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:39.448632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:39.448632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:39.448632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:39.448632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:39.448632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:39.448632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:39.448632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:39.449634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:39.449634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:39.449634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:39.449634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:39.449634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:39.449634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:39.449634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:39.449634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:39.449634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:39.449634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:39.449634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:39.449634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:39.449634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:39.449634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:39.449634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:39.449634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:39.449634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:39.450587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:39.450587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:39.450587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:39.450587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:39.450587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 340/678] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:39.450869 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49fbb3-7c03-4f58-a5d1-77a541903f47 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:39.459441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98af745-3998-4bfe-a4ef-59f6e3352a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:39.461184 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98af745-3998-4bfe-a4ef-59f6e3352a56 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:39.461184 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98af745-3998-4bfe-a4ef-59f6e3352a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:39.461184 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98af745-3998-4bfe-a4ef-59f6e3352a56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:39.461537 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98af745-3998-4bfe-a4ef-59f6e3352a56 None admin] RBAC: Adding query filter params (role.id=0e34a6f9b89a4e29951fb189abacb34a, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=1fadecafbe2549acaa4cde5412638d8f) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:39.462528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98af745-3998-4bfe-a4ef-59f6e3352a56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:39.463255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98af745-3998-4bfe-a4ef-59f6e3352a56 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:39.470172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 339/679] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:21:39 2022] GET /identity/v3/role_assignments?user.id=1fadecafbe2549acaa4cde5412638d8f&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=0e34a6f9b89a4e29951fb189abacb34a => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:39.471007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98af745-3998-4bfe-a4ef-59f6e3352a56 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:39.540065 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:39.560528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:39.596508 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49fbb3-7c03-4f58-a5d1-77a541903f47 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:39.624765 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98af745-3998-4bfe-a4ef-59f6e3352a56 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:39.636014 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a518d-9e5b-44db-ae2a-dec8fc0a601f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:39.636863 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a518d-9e5b-44db-ae2a-dec8fc0a601f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:39.637549 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a518d-9e5b-44db-ae2a-dec8fc0a601f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:39.637549 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a518d-9e5b-44db-ae2a-dec8fc0a601f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:39.650148 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a518d-9e5b-44db-ae2a-dec8fc0a601f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:39.657920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-b33a518d-9e5b-44db-ae2a-dec8fc0a601f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:21:39.657920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:39.657920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:39.657920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:39.657920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:39.657920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:39.657920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:39.657920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:39.657920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:39.657920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:39.657920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:39.657920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:39.657920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:39.657920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:39.657920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:39.657920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:39.659140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:39.659140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:39.659140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:39.659140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:39.659140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:39.659140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:39.659140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:39.659140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:39.659140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:39.659140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:39.659140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:39.659140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:39.659140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:39.659140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:39.659140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:39.659140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:39.660194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:39.660194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:39.660194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:39.660194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:39.660194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:39.660194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:39.660194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:39.660194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:39.660194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:39.660194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:39.660194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:21:39.660194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:39.660194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 341/680] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:21:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:39.661008 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a518d-9e5b-44db-ae2a-dec8fc0a601f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:39.683197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf4081-c9d2-4a58-84a6-7bba685a677e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:39.683880 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf4081-c9d2-4a58-84a6-7bba685a677e None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:39.684173 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf4081-c9d2-4a58-84a6-7bba685a677e None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:39.684457 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf4081-c9d2-4a58-84a6-7bba685a677e None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:39.685301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf4081-c9d2-4a58-84a6-7bba685a677e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:39.685620 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf4081-c9d2-4a58-84a6-7bba685a677e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:39.687007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf4081-c9d2-4a58-84a6-7bba685a677e None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:39.702302 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 340/681] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:39.703334 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf4081-c9d2-4a58-84a6-7bba685a677e None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:39.799513 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a518d-9e5b-44db-ae2a-dec8fc0a601f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:39.834071 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:39.838653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf4081-c9d2-4a58-84a6-7bba685a677e None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:39.845443 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e1edc-dbb4-4500-9c72-6903c76a8a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:39.846205 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e1edc-dbb4-4500-9c72-6903c76a8a0c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:39.846496 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e1edc-dbb4-4500-9c72-6903c76a8a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:39.846777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e1edc-dbb4-4500-9c72-6903c76a8a0c None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:39.847530 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e1edc-dbb4-4500-9c72-6903c76a8a0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:39.847848 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e1edc-dbb4-4500-9c72-6903c76a8a0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:39.849267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e1edc-dbb4-4500-9c72-6903c76a8a0c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:39.869238 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 342/682] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:39.869585 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:39.870720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e1edc-dbb4-4500-9c72-6903c76a8a0c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:39.886309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fffc13-68a2-4dc2-811e-f1a4612a2b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:39.887400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fffc13-68a2-4dc2-811e-f1a4612a2b4f None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:39.887951 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fffc13-68a2-4dc2-811e-f1a4612a2b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:39.888453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fffc13-68a2-4dc2-811e-f1a4612a2b4f None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:39.889473 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fffc13-68a2-4dc2-811e-f1a4612a2b4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:39.889995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fffc13-68a2-4dc2-811e-f1a4612a2b4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:39.892429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fffc13-68a2-4dc2-811e-f1a4612a2b4f None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:39.902515 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 341/683] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:39.903825 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fffc13-68a2-4dc2-811e-f1a4612a2b4f None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:39.990009 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e1edc-dbb4-4500-9c72-6903c76a8a0c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:40.027508 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00337ee1-795e-4f7c-b537-6e297e18cc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:40.028835 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00337ee1-795e-4f7c-b537-6e297e18cc50 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:40.029367 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00337ee1-795e-4f7c-b537-6e297e18cc50 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:40.029964 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00337ee1-795e-4f7c-b537-6e297e18cc50 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:40.038632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00337ee1-795e-4f7c-b537-6e297e18cc50 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:40.040659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00337ee1-795e-4f7c-b537-6e297e18cc50 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:40.040849 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fffc13-68a2-4dc2-811e-f1a4612a2b4f None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:40.058818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-00337ee1-795e-4f7c-b537-6e297e18cc50 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:21:40.058818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:40.058818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:40.058818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:40.058818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:40.058818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:40.058818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:40.058818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:40.058818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:40.058818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:40.058818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:40.058818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:40.058818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:40.058818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:40.058818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:40.058818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:40.059850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:40.059850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:40.059850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:40.059850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:40.059850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:40.059850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:40.059850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:40.059850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:40.059850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:40.059850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:40.059850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:40.059850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:40.059850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:40.059850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:40.059850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:40.059850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:40.059850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:40.059850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:40.060841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:40.060841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:40.060841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:40.060841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:40.060841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:40.060841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:40.060841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:40.060841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:40.060841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:21:40.060841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:40.069329 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 343/684] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:21:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:40.070950 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00337ee1-795e-4f7c-b537-6e297e18cc50 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:40.080567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328f5c5-77ff-4fbd-bc20-8da3d45b0e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:40.081416 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328f5c5-77ff-4fbd-bc20-8da3d45b0e71 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:40.081755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328f5c5-77ff-4fbd-bc20-8da3d45b0e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:40.082038 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328f5c5-77ff-4fbd-bc20-8da3d45b0e71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:40.090680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328f5c5-77ff-4fbd-bc20-8da3d45b0e71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:40.092597 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328f5c5-77ff-4fbd-bc20-8da3d45b0e71 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:40.114194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-9328f5c5-77ff-4fbd-bc20-8da3d45b0e71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:40.114194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:40.114194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:40.114194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:40.114194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:40.114194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:40.114194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:40.114194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:40.114194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:40.114194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:40.114194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:40.114194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:40.114194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:40.114194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:40.114194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:40.115186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:40.115186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:40.115186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:40.115186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:40.115186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:40.115186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:40.115186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:40.115186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:40.115186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:40.115186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:40.115186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:40.115186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:40.115186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:40.115186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:40.115186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:40.115186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:40.115186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:40.116102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:40.116102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:40.116102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:40.116102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:40.116102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:40.116102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:40.116102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:40.116102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:40.120221 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 342/685] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:40.121288 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328f5c5-77ff-4fbd-bc20-8da3d45b0e71 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:40.147356 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:40.175632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:40.221416 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00337ee1-795e-4f7c-b537-6e297e18cc50 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:40.251428 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328f5c5-77ff-4fbd-bc20-8da3d45b0e71 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:40.268766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864375a0-5e12-4fc2-b32a-b72b9af57988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:40.269934 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864375a0-5e12-4fc2-b32a-b72b9af57988 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:40.270484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864375a0-5e12-4fc2-b32a-b72b9af57988 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:40.270970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864375a0-5e12-4fc2-b32a-b72b9af57988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:40.282958 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864375a0-5e12-4fc2-b32a-b72b9af57988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:40.284796 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864375a0-5e12-4fc2-b32a-b72b9af57988 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:40.296236 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cfef8-38ba-4031-b347-9f7131f2c848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:40.296947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cfef8-38ba-4031-b347-9f7131f2c848 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:40.297286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cfef8-38ba-4031-b347-9f7131f2c848 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:40.297569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cfef8-38ba-4031-b347-9f7131f2c848 None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:40.298425 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81cfef8-38ba-4031-b347-9f7131f2c848 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:40.298742 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81cfef8-38ba-4031-b347-9f7131f2c848 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:40.300169 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81cfef8-38ba-4031-b347-9f7131f2c848 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:40.305332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-864375a0-5e12-4fc2-b32a-b72b9af57988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:40.305332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:40.305332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:40.305332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:40.305332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:40.305332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:40.305332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:40.305332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:40.305332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:40.305332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:40.305332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:40.305332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:40.305332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:40.305332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:40.305332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:40.306231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:40.306231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:40.306231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:40.306231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:40.306231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:40.306231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:40.306231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:40.306231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:40.306231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:40.306231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:40.306231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:40.306231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:40.306231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:40.306231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:40.306231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:40.306231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:40.306231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:40.307156 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:40.307156 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:40.307156 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:40.307156 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:40.309437 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 344/686] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:40.310512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864375a0-5e12-4fc2-b32a-b72b9af57988 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:40.324939 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 343/687] 10.222.0.40 () {56 vars in 1135 bytes} [Thu Jun 16 17:21:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:40.324939 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cfef8-38ba-4031-b347-9f7131f2c848 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:40.431980 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:40.436092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864375a0-5e12-4fc2-b32a-b72b9af57988 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:40.459669 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:40.463075 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cfef8-38ba-4031-b347-9f7131f2c848 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:40.475053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64185ed-2414-450b-af4f-e442b033064d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:40.476037 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64185ed-2414-450b-af4f-e442b033064d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:40.476553 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64185ed-2414-450b-af4f-e442b033064d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:40.477070 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64185ed-2414-450b-af4f-e442b033064d None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:40.478102 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64185ed-2414-450b-af4f-e442b033064d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:40.478622 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64185ed-2414-450b-af4f-e442b033064d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:40.480255 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64185ed-2414-450b-af4f-e442b033064d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:40.492959 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 345/688] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:40.494112 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64185ed-2414-450b-af4f-e442b033064d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:40.506927 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5159e-15db-4ca3-b9fe-8f039e51634b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:40.507889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5159e-15db-4ca3-b9fe-8f039e51634b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:40.508107 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5159e-15db-4ca3-b9fe-8f039e51634b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:40.508324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5159e-15db-4ca3-b9fe-8f039e51634b None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:40.509477 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d5159e-15db-4ca3-b9fe-8f039e51634b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:40.509769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d5159e-15db-4ca3-b9fe-8f039e51634b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:40.511298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d5159e-15db-4ca3-b9fe-8f039e51634b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:40.525130 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 344/689] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:40.525984 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5159e-15db-4ca3-b9fe-8f039e51634b None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:40.604830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64185ed-2414-450b-af4f-e442b033064d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:40.641620 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5159e-15db-4ca3-b9fe-8f039e51634b None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:40.645190 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac724576-cb8f-4dec-a6ff-b00de24fbe68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:40.646122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac724576-cb8f-4dec-a6ff-b00de24fbe68 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:40.646616 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac724576-cb8f-4dec-a6ff-b00de24fbe68 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:40.647121 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac724576-cb8f-4dec-a6ff-b00de24fbe68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:40.677792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:40.681745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6070eef-e272-40c0-b385-1dd0ca64cd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:40.682496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6070eef-e272-40c0-b385-1dd0ca64cd42 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:40.682768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6070eef-e272-40c0-b385-1dd0ca64cd42 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:40.683103 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6070eef-e272-40c0-b385-1dd0ca64cd42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:40.683958 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6070eef-e272-40c0-b385-1dd0ca64cd42 None admin] RBAC: Adding query filter params (role.id=0e34a6f9b89a4e29951fb189abacb34a, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=a303489493ce4cd2b62e876fbc8a7404) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:40.684326 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6070eef-e272-40c0-b385-1dd0ca64cd42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:40.686451 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6070eef-e272-40c0-b385-1dd0ca64cd42 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:40.701648 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 345/690] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:21:40 2022] GET /identity/v3/role_assignments?user.id=a303489493ce4cd2b62e876fbc8a7404&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=0e34a6f9b89a4e29951fb189abacb34a => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:40.702633 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6070eef-e272-40c0-b385-1dd0ca64cd42 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:40.725559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac724576-cb8f-4dec-a6ff-b00de24fbe68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:40.727387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac724576-cb8f-4dec-a6ff-b00de24fbe68 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:40.752224 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:40.847955 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 346/691] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:21:40.855321 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac724576-cb8f-4dec-a6ff-b00de24fbe68 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:40.927315 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:40.965940 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6070eef-e272-40c0-b385-1dd0ca64cd42 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:40.973744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac724576-cb8f-4dec-a6ff-b00de24fbe68 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:41.023752 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a644a-76a1-49b9-bce5-19323039b23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:41.024192 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c43be-c930-4c0a-acb0-c3a91d877203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:41.024561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a644a-76a1-49b9-bce5-19323039b23c None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:41.024850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a644a-76a1-49b9-bce5-19323039b23c None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:41.025133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a644a-76a1-49b9-bce5-19323039b23c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:41.025928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c43be-c930-4c0a-acb0-c3a91d877203 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:41.026433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c43be-c930-4c0a-acb0-c3a91d877203 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:41.026925 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c43be-c930-4c0a-acb0-c3a91d877203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:41.038498 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a644a-76a1-49b9-bce5-19323039b23c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:41.039955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a644a-76a1-49b9-bce5-19323039b23c None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:41.044976 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c43be-c930-4c0a-acb0-c3a91d877203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:41.046712 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c43be-c930-4c0a-acb0-c3a91d877203 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:41.059983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-131a644a-76a1-49b9-bce5-19323039b23c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:21:41.059983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:41.059983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:41.059983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:41.059983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:41.059983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:41.059983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:41.059983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:41.059983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:41.059983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:41.059983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:41.059983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:41.059983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:41.059983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:41.059983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:41.059983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:41.061068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:41.061068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:41.061068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:41.061068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:41.061068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:41.061068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:41.061068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:41.061068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:41.061068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:41.061068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:41.061068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:41.061068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:41.061068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:41.061068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:41.061068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:41.061068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:41.061068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:41.061068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:41.062118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:41.062118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:41.062118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:41.062118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:41.062118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:41.062118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:41.062118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:41.062118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:41.062118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:21:41.062118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:41.062118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 346/692] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jun 16 17:21:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:41.067906 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a644a-76a1-49b9-bce5-19323039b23c None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:41.072642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-a14c43be-c930-4c0a-acb0-c3a91d877203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:41.072642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:41.072642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:41.072642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:41.072642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:41.072642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:41.072642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:41.072642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:41.072642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:41.072642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:41.072642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:41.072642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:41.072642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:41.072642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:41.072642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:41.073619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:41.073619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:41.073619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:41.073619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:41.073619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:41.073619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:41.073619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:41.073619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:41.073619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:41.073619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:41.073619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:41.073619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:41.073619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:41.073619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:41.073619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:41.073619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:41.073619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:41.074547 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:41.074547 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:41.074547 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:41.074547 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:41.076895 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 347/693] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:41.083569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c43be-c930-4c0a-acb0-c3a91d877203 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:41.170350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:41.192667 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a644a-76a1-49b9-bce5-19323039b23c None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:41.214944 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c43be-c930-4c0a-acb0-c3a91d877203 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:41.236496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb42402-73f9-44fd-994e-63a63e5c11a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:41.237040 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb42402-73f9-44fd-994e-63a63e5c11a3 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:41.237479 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb42402-73f9-44fd-994e-63a63e5c11a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:41.237775 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb42402-73f9-44fd-994e-63a63e5c11a3 None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:41.238651 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb42402-73f9-44fd-994e-63a63e5c11a3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:41.238972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb42402-73f9-44fd-994e-63a63e5c11a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:41.240445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb42402-73f9-44fd-994e-63a63e5c11a3 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:41.245984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:41.262763 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 347/694] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jun 16 17:21:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:41.269717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb42402-73f9-44fd-994e-63a63e5c11a3 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:41.290860 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428240d-3913-45f9-bc4c-cb58c98991e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:41.291843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428240d-3913-45f9-bc4c-cb58c98991e7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:41.292310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428240d-3913-45f9-bc4c-cb58c98991e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:41.292754 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428240d-3913-45f9-bc4c-cb58c98991e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:41.293694 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428240d-3913-45f9-bc4c-cb58c98991e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:41.294158 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428240d-3913-45f9-bc4c-cb58c98991e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:41.295713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428240d-3913-45f9-bc4c-cb58c98991e7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:41.323744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 348/695] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:41.334224 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428240d-3913-45f9-bc4c-cb58c98991e7 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:41.386672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb42402-73f9-44fd-994e-63a63e5c11a3 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:41.429572 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ff872-9738-4a7c-91e9-7013709eef88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:41.430158 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ff872-9738-4a7c-91e9-7013709eef88 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:41.430482 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ff872-9738-4a7c-91e9-7013709eef88 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:41.430768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ff872-9738-4a7c-91e9-7013709eef88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:41.461516 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:41.540386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ff872-9738-4a7c-91e9-7013709eef88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:41.541911 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ff872-9738-4a7c-91e9-7013709eef88 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:41.623415 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428240d-3913-45f9-bc4c-cb58c98991e7 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:41.662044 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 348/696] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:21:41.667374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ff872-9738-4a7c-91e9-7013709eef88 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:41.679655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a28f6-24d9-40ce-99ea-491321a76cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:41.680408 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a28f6-24d9-40ce-99ea-491321a76cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:41.680699 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a28f6-24d9-40ce-99ea-491321a76cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:41.680982 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a28f6-24d9-40ce-99ea-491321a76cb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:41.695643 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a28f6-24d9-40ce-99ea-491321a76cb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:41.697880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a28f6-24d9-40ce-99ea-491321a76cb7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:41.718398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-ad4a28f6-24d9-40ce-99ea-491321a76cb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:41.718398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:41.718398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:41.718398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:41.718398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:41.718398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:41.718398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:41.718398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:41.718398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:41.718398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:41.718398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:41.718398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:41.718398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:41.718398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:41.718398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:41.720361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:41.720361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:41.720361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:41.720361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:41.720361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:41.720361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:41.720361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:41.720361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:41.720361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:41.720361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:41.720361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:41.720361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:41.720361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:41.720361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:41.720361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:41.720361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:41.720361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:41.721303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:41.721303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:41.721303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:41.721303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:41.721303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:41.721303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:41.721303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:41.721303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:41.721303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 349/697] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:41.722293 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcaaf1-712e-45f7-b47c-abec222475e0 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:41.722293 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcaaf1-712e-45f7-b47c-abec222475e0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:41.722593 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcaaf1-712e-45f7-b47c-abec222475e0 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:41.727316 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 350/698] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:41.728385 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcaaf1-712e-45f7-b47c-abec222475e0 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:41.735177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:41.772148 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:41.803248 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ff872-9738-4a7c-91e9-7013709eef88 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:41.854003 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99caf6b-2442-43aa-af8b-f32feec15b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:41.855182 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99caf6b-2442-43aa-af8b-f32feec15b81 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:41.855470 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99caf6b-2442-43aa-af8b-f32feec15b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:41.855750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99caf6b-2442-43aa-af8b-f32feec15b81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:41.868084 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99caf6b-2442-43aa-af8b-f32feec15b81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:41.869562 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99caf6b-2442-43aa-af8b-f32feec15b81 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:41.872969 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcaaf1-712e-45f7-b47c-abec222475e0 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:41.891376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-b99caf6b-2442-43aa-af8b-f32feec15b81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:41.891376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:41.891376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:41.891376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:41.891376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:41.891376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:41.891376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:41.891376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:41.891376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:41.891376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:41.891376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:41.891376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:41.891376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:41.891376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:41.891376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:41.892463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:41.892463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:41.892463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:41.892463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:41.892463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:41.892463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:41.892463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:41.892463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:41.892463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:41.892463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:41.892463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:41.892463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:41.892463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:41.892463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:41.892463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:41.892463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:41.892463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:41.893425 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:41.893425 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:41.893425 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:41.893425 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:41.897107 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 349/699] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:41.898352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8193d3-1d26-42de-8ab7-49c022be0d19 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:41.898645 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8193d3-1d26-42de-8ab7-49c022be0d19 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:41.898927 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8193d3-1d26-42de-8ab7-49c022be0d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:41.913356 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54a185-fbad-47a9-b84f-2418f120fd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:41.914789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54a185-fbad-47a9-b84f-2418f120fd02 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:41.915220 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54a185-fbad-47a9-b84f-2418f120fd02 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:41.915638 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54a185-fbad-47a9-b84f-2418f120fd02 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:41.916272 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-8f8193d3-1d26-42de-8ab7-49c022be0d19 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:41.916730 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54a185-fbad-47a9-b84f-2418f120fd02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:41.917224 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54a185-fbad-47a9-b84f-2418f120fd02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:41.917487 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-8f8193d3-1d26-42de-8ab7-49c022be0d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:41.918962 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54a185-fbad-47a9-b84f-2418f120fd02 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:41.932404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 351/700] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:41.933908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e517e8-dc1d-4d21-9a82-e17fb2d524e7 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:41.934199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e517e8-dc1d-4d21-9a82-e17fb2d524e7 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:41.934480 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e517e8-dc1d-4d21-9a82-e17fb2d524e7 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:41.935958 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 352/701] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:41.936519 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e517e8-dc1d-4d21-9a82-e17fb2d524e7 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:42.003266 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 350/702] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:42.004832 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8193d3-1d26-42de-8ab7-49c022be0d19 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:42.040467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:42.053396 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:42.064920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e517e8-dc1d-4d21-9a82-e17fb2d524e7 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:42.102668 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a17a7-8cf8-4763-b69b-e6e7ac43f4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:42.103446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a17a7-8cf8-4763-b69b-e6e7ac43f4eb None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:42.103729 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a17a7-8cf8-4763-b69b-e6e7ac43f4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:42.104007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a17a7-8cf8-4763-b69b-e6e7ac43f4eb None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:42.104744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0a17a7-8cf8-4763-b69b-e6e7ac43f4eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:42.105041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0a17a7-8cf8-4763-b69b-e6e7ac43f4eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:42.106531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0a17a7-8cf8-4763-b69b-e6e7ac43f4eb None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:42.118022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8193d3-1d26-42de-8ab7-49c022be0d19 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:42.123073 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 353/703] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:42.123073 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946e3e9-4d46-49c5-91a2-71439d6da218 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:42.123073 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946e3e9-4d46-49c5-91a2-71439d6da218 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:42.123073 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946e3e9-4d46-49c5-91a2-71439d6da218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:42.138942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4946e3e9-4d46-49c5-91a2-71439d6da218 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:42.140233 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4946e3e9-4d46-49c5-91a2-71439d6da218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:42.158614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b50f3-f9e7-4b04-bb28-aadec4bbca33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:42.159377 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b50f3-f9e7-4b04-bb28-aadec4bbca33 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:42.159687 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b50f3-f9e7-4b04-bb28-aadec4bbca33 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:42.159948 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b50f3-f9e7-4b04-bb28-aadec4bbca33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:42.160779 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b50f3-f9e7-4b04-bb28-aadec4bbca33 None admin] RBAC: Adding query filter params (role.id=0e34a6f9b89a4e29951fb189abacb34a, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=b8a37d1201d54afdad9b228ef4f2d787) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:42.161092 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b50f3-f9e7-4b04-bb28-aadec4bbca33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:42.162536 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b50f3-f9e7-4b04-bb28-aadec4bbca33 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:42.177183 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 351/704] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:21:42 2022] GET /identity/v3/role_assignments?user.id=b8a37d1201d54afdad9b228ef4f2d787&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=0e34a6f9b89a4e29951fb189abacb34a => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:42.177183 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b50f3-f9e7-4b04-bb28-aadec4bbca33 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:42.247052 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 354/705] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:42.248038 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4946e3e9-4d46-49c5-91a2-71439d6da218 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:42.291201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:42.307836 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b50f3-f9e7-4b04-bb28-aadec4bbca33 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:42.350244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936a85c-c9c6-4452-ba63-99a9cfb76589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:42.351268 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936a85c-c9c6-4452-ba63-99a9cfb76589 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:42.351654 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936a85c-c9c6-4452-ba63-99a9cfb76589 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:42.352003 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936a85c-c9c6-4452-ba63-99a9cfb76589 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:42.360673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:42.364171 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1936a85c-c9c6-4452-ba63-99a9cfb76589 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:42.365810 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1936a85c-c9c6-4452-ba63-99a9cfb76589 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:42.386596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-1936a85c-c9c6-4452-ba63-99a9cfb76589 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:42.386596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:42.386596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:42.386596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:42.386596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:42.386596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:42.386596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:42.386596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:42.386596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:42.386596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:42.386596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:42.386596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:42.386596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:42.386596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:42.386596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:42.387681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:42.387681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:42.387681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:42.387681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:42.387681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:42.387681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:42.387681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:42.387681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:42.387681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:42.387681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:42.387681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:42.387681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:42.387681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:42.387681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:42.387681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:42.387681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:42.387681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:42.388665 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:42.388665 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:42.388665 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:42.388665 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:42.393094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4946e3e9-4d46-49c5-91a2-71439d6da218 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:42.393427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 352/706] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:42.394445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936a85c-c9c6-4452-ba63-99a9cfb76589 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:42.434584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478cd959-f686-4a18-99d5-c8d25348a0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:42.435644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cd959-f686-4a18-99d5-c8d25348a0bf None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:42.436017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cd959-f686-4a18-99d5-c8d25348a0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:42.436365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cd959-f686-4a18-99d5-c8d25348a0bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:42.446336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478cd959-f686-4a18-99d5-c8d25348a0bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:42.448007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478cd959-f686-4a18-99d5-c8d25348a0bf None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:42.465884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-478cd959-f686-4a18-99d5-c8d25348a0bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:42.465884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:42.465884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:42.465884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:42.465884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:42.465884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:42.465884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:42.465884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:42.465884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:42.465884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:42.465884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:42.465884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:42.465884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:42.465884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:42.465884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:42.466899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:42.466899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:42.466899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:42.466899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:42.466899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:42.466899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:42.466899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:42.466899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:42.466899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:42.466899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:42.466899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:42.466899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:42.466899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:42.466899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:42.466899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:42.466899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:42.466899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:42.467866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:42.467866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:42.467866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:42.467866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:42.467866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:42.467866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:42.467866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:42.467866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:42.467866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 355/707] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:42.468379 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478cd959-f686-4a18-99d5-c8d25348a0bf None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:42.516870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936a85c-c9c6-4452-ba63-99a9cfb76589 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:42.558805 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677ce96-8af0-4463-9a39-d485de2cd0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:42.560118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677ce96-8af0-4463-9a39-d485de2cd0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:42.560683 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677ce96-8af0-4463-9a39-d485de2cd0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:42.560970 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677ce96-8af0-4463-9a39-d485de2cd0f7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:42.561888 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677ce96-8af0-4463-9a39-d485de2cd0f7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:42.564718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677ce96-8af0-4463-9a39-d485de2cd0f7 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:42.582718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-9677ce96-8af0-4463-9a39-d485de2cd0f7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:21:42.582718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:42.582718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:21:42.582718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:21:42.582718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:21:42.582718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:21:42.582718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:21:42.582718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:21:42.582718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:21:42.582718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:42.582718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:21:42.582718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:42.582718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:42.582718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:42.582718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:42.582718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:42.583768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:42.583768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:42.583768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:42.583768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:42.583768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:42.583768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:42.583768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:42.583768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:21:42.583768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:21:42.583768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:21:42.583768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:21:42.583768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:42.583768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:42.583768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:42.583768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:42.583768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:42.585126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:42.585126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:42.585126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:42.585126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:42.585126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:42.585126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:42.585126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:42.585126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:42.585126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:42.585126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:42.585126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:42.585126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:21:42.585126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:21:42.585126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:21:42.585126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:42.585126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 353/708] 10.222.0.40 () {56 vars in 1129 bytes} [Thu Jun 16 17:21:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:42.586423 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677ce96-8af0-4463-9a39-d485de2cd0f7 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:42.591554 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:42.602954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478cd959-f686-4a18-99d5-c8d25348a0bf None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:42.655597 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d79c1-4e2b-4542-a8da-4d43545eb2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:42.656769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d79c1-4e2b-4542-a8da-4d43545eb2ba None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:42.657059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d79c1-4e2b-4542-a8da-4d43545eb2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:42.657516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d79c1-4e2b-4542-a8da-4d43545eb2ba None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:42.658293 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d79c1-4e2b-4542-a8da-4d43545eb2ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:42.658607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d79c1-4e2b-4542-a8da-4d43545eb2ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:42.660546 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d79c1-4e2b-4542-a8da-4d43545eb2ba None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:42.683969 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 356/709] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:42.684966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d79c1-4e2b-4542-a8da-4d43545eb2ba None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:42.688562 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:42.733348 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677ce96-8af0-4463-9a39-d485de2cd0f7 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:42.771436 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af996f5-6fbf-421d-8124-d0d56f25e744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:42.772180 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af996f5-6fbf-421d-8124-d0d56f25e744 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:42.772572 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af996f5-6fbf-421d-8124-d0d56f25e744 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:42.772923 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af996f5-6fbf-421d-8124-d0d56f25e744 None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:42.773826 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af996f5-6fbf-421d-8124-d0d56f25e744 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:42.774209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af996f5-6fbf-421d-8124-d0d56f25e744 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:42.775731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af996f5-6fbf-421d-8124-d0d56f25e744 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:42.789477 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 354/710] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:42.790411 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af996f5-6fbf-421d-8124-d0d56f25e744 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:42.831237 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d79c1-4e2b-4542-a8da-4d43545eb2ba None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:42.862238 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:42.879065 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c4da74-d8ed-4df8-9d91-88195da2405e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:42.880185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4da74-d8ed-4df8-9d91-88195da2405e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:42.880473 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4da74-d8ed-4df8-9d91-88195da2405e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:42.881165 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4da74-d8ed-4df8-9d91-88195da2405e None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:42.882176 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c4da74-d8ed-4df8-9d91-88195da2405e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:42.882498 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c4da74-d8ed-4df8-9d91-88195da2405e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:42.884381 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c4da74-d8ed-4df8-9d91-88195da2405e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:42.895631 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 357/711] 10.222.0.40 () {56 vars in 1124 bytes} [Thu Jun 16 17:21:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:42.896612 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c4da74-d8ed-4df8-9d91-88195da2405e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:42.907772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af996f5-6fbf-421d-8124-d0d56f25e744 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:42.945319 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376f6d2-cb36-4b1d-8ff3-466bb5d1ed34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:42.946027 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376f6d2-cb36-4b1d-8ff3-466bb5d1ed34 None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:42.946318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376f6d2-cb36-4b1d-8ff3-466bb5d1ed34 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:42.946603 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376f6d2-cb36-4b1d-8ff3-466bb5d1ed34 None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:42.947380 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376f6d2-cb36-4b1d-8ff3-466bb5d1ed34 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:42.948820 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376f6d2-cb36-4b1d-8ff3-466bb5d1ed34 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:42.964944 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:42.973270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 355/712] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 17:21:42 2022] POST /identity/v3/users => generated 254 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:21:42.974227 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376f6d2-cb36-4b1d-8ff3-466bb5d1ed34 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:43.005020 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c4da74-d8ed-4df8-9d91-88195da2405e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:43.043487 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08a1d9-d49b-4956-8dc9-f2e2a62a9cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:43.044527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08a1d9-d49b-4956-8dc9-f2e2a62a9cdd None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:43.044960 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08a1d9-d49b-4956-8dc9-f2e2a62a9cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:43.045413 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08a1d9-d49b-4956-8dc9-f2e2a62a9cdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:43.046545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08a1d9-d49b-4956-8dc9-f2e2a62a9cdd None admin] RBAC: Adding query filter params (role.id=25a16d8ea4084e9eacdebb7a4dcd2337, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=689a31e26c6b454a91c3fd79678b4d07) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:43.047579 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08a1d9-d49b-4956-8dc9-f2e2a62a9cdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:43.049213 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08a1d9-d49b-4956-8dc9-f2e2a62a9cdd None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:43.056094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 358/713] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:21:42 2022] GET /identity/v3/role_assignments?user.id=689a31e26c6b454a91c3fd79678b4d07&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=25a16d8ea4084e9eacdebb7a4dcd2337 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:43.097240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376f6d2-cb36-4b1d-8ff3-466bb5d1ed34 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:43.122741 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:43.143172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a9ba1-cf91-4639-87da-1d61a7f0ada7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:43.143869 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a9ba1-cf91-4639-87da-1d61a7f0ada7 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:43.144160 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a9ba1-cf91-4639-87da-1d61a7f0ada7 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:43.144444 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a9ba1-cf91-4639-87da-1d61a7f0ada7 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:43.145203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a9ba1-cf91-4639-87da-1d61a7f0ada7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:43.145523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a9ba1-cf91-4639-87da-1d61a7f0ada7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:43.146929 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a9ba1-cf91-4639-87da-1d61a7f0ada7 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:43.154086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 356/714] 10.222.0.40 () {56 vars in 1124 bytes} [Thu Jun 16 17:21:42 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:43.580425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd752d1-9abc-4771-98d3-1203230bee14 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:43.581036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd752d1-9abc-4771-98d3-1203230bee14 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:43.581036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd752d1-9abc-4771-98d3-1203230bee14 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:43.582738 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 359/715] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:43.588332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d56421-9814-487b-9768-42c29577ed17 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:43.588635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d56421-9814-487b-9768-42c29577ed17 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:43.588926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d56421-9814-487b-9768-42c29577ed17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:43.608182 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-63d56421-9814-487b-9768-42c29577ed17 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:43.609475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-63d56421-9814-487b-9768-42c29577ed17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:43.707444 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 357/716] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:43.743183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd752d1-9abc-4771-98d3-1203230bee14 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:43.888646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd752d1-9abc-4771-98d3-1203230bee14 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:43.929043 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:43.938039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e71f7-e50c-42c7-a15d-a03cea0934ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:43.939004 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e71f7-e50c-42c7-a15d-a03cea0934ba None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:43.939291 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e71f7-e50c-42c7-a15d-a03cea0934ba None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:43.939571 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e71f7-e50c-42c7-a15d-a03cea0934ba None admin] PATH_INFO: `/v3/services/network` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:43.940237 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e71f7-e50c-42c7-a15d-a03cea0934ba None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:43.941656 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e71f7-e50c-42c7-a15d-a03cea0934ba None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:43.956204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-af8e71f7-e50c-42c7-a15d-a03cea0934ba None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:21:43.956204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:43.956204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:21:43.956204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:21:43.956204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:21:43.956204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:21:43.956204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:21:43.956204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:21:43.956204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:21:43.956204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:43.956204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:21:43.956204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:43.956204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:43.956204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:43.956204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:43.956204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:43.956204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:43.957288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:43.957288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:43.957288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:43.957288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:43.957288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:43.957288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:43.957288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:21:43.957288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:21:43.957288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:21:43.957288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:21:43.957288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:43.957288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:43.957288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:43.957288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:43.957288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:43.957288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:43.958178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:43.958178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:43.958178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:43.958178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:43.958178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:43.958178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:43.958178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:43.958178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:43.958178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:43.958178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:43.958178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:21:43.958178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:21:43.958178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:21:43.958178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:43.958178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 360/717] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:43 2022] GET /identity/v3/services/network => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:43.963807 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d56421-9814-487b-9768-42c29577ed17 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:44.127430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d56421-9814-487b-9768-42c29577ed17 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:44.144579 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:44.173851 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb74b4-9010-4519-98a3-be1864eaa4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:44.174532 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb74b4-9010-4519-98a3-be1864eaa4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:44.174826 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb74b4-9010-4519-98a3-be1864eaa4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:44.175112 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb74b4-9010-4519-98a3-be1864eaa4f4 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:44.175832 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb74b4-9010-4519-98a3-be1864eaa4f4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:44.176142 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb74b4-9010-4519-98a3-be1864eaa4f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:44.177827 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb74b4-9010-4519-98a3-be1864eaa4f4 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:44.182919 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 358/718] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 17:21:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:44.188179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e71f7-e50c-42c7-a15d-a03cea0934ba None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:44.323689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e71f7-e50c-42c7-a15d-a03cea0934ba None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:44.358013 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b3ccd-b8b7-465d-a459-4e0aa4984526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:44.358685 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b3ccd-b8b7-465d-a459-4e0aa4984526 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:44.358969 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b3ccd-b8b7-465d-a459-4e0aa4984526 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:44.359285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b3ccd-b8b7-465d-a459-4e0aa4984526 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:44.360187 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b3ccd-b8b7-465d-a459-4e0aa4984526 None admin] RBAC: Adding query filter params (type=network) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:44.360500 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b3ccd-b8b7-465d-a459-4e0aa4984526 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:44.361887 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b3ccd-b8b7-465d-a459-4e0aa4984526 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:44.367959 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 361/719] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 17:21:44 2022] GET /identity/v3/services?type=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:44.982009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d330f-e439-46a6-ba63-35a84e079291 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:44.982517 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d330f-e439-46a6-ba63-35a84e079291 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:44.982908 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d330f-e439-46a6-ba63-35a84e079291 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:44.984633 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 359/720] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:44.990275 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd6b5b-16ce-4887-91f3-a5077a9026a2 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:44.990724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd6b5b-16ce-4887-91f3-a5077a9026a2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:44.990904 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd6b5b-16ce-4887-91f3-a5077a9026a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:45.008815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-ebbd6b5b-16ce-4887-91f3-a5077a9026a2 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:45.010044 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ebbd6b5b-16ce-4887-91f3-a5077a9026a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:45.065100 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:45.079758 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 362/721] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:21:45.102680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d330f-e439-46a6-ba63-35a84e079291 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:45.247075 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d330f-e439-46a6-ba63-35a84e079291 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:45.285760 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04318e-bd5e-47af-a843-b58a03c29de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:45.286574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04318e-bd5e-47af-a843-b58a03c29de3 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:45.286862 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04318e-bd5e-47af-a843-b58a03c29de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:45.287144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04318e-bd5e-47af-a843-b58a03c29de3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:45.298550 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04318e-bd5e-47af-a843-b58a03c29de3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:45.306450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-df04318e-bd5e-47af-a843-b58a03c29de3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:21:45.306450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:45.306450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:45.306450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:45.306450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:45.306450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:45.306450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:45.306450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:45.306450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:45.306450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:45.306450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:45.306450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:45.306450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:45.306450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:45.306450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:45.306450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:45.307426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:45.307426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:45.307426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:45.307426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:45.307426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:45.307426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:45.307426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:45.307426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:45.307426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:45.307426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:45.307426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:45.307426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:45.307426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:45.307426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:45.307426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:45.307426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:45.309270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:45.309270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:45.309270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:45.309270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:45.309270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:45.309270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:45.309270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:45.309270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:45.309270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:45.309270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:45.309270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:21:45.309270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:45.309270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 360/722] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:21:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:45.314378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd6b5b-16ce-4887-91f3-a5077a9026a2 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:45.418798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd6b5b-16ce-4887-91f3-a5077a9026a2 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:45.455083 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721f6c9-5603-4915-b638-c66add4a5528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:45.455775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721f6c9-5603-4915-b638-c66add4a5528 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:45.456064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721f6c9-5603-4915-b638-c66add4a5528 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:45.456354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721f6c9-5603-4915-b638-c66add4a5528 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:45.457118 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721f6c9-5603-4915-b638-c66add4a5528 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:45.457472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721f6c9-5603-4915-b638-c66add4a5528 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:45.458924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721f6c9-5603-4915-b638-c66add4a5528 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:45.468169 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 363/723] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:45.473844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04318e-bd5e-47af-a843-b58a03c29de3 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:45.517348 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:45.592026 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04318e-bd5e-47af-a843-b58a03c29de3 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:45.641809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb13de0-a74f-48df-a47c-0336328d640a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:45.642681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb13de0-a74f-48df-a47c-0336328d640a None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:45.642976 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb13de0-a74f-48df-a47c-0336328d640a None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:45.643327 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb13de0-a74f-48df-a47c-0336328d640a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:45.658071 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb13de0-a74f-48df-a47c-0336328d640a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:45.659537 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb13de0-a74f-48df-a47c-0336328d640a None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:45.677591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-7eb13de0-a74f-48df-a47c-0336328d640a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:45.677591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:45.677591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:45.677591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:45.677591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:45.677591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:45.677591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:45.677591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:45.677591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:45.677591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:45.677591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:45.677591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:45.677591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:45.677591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:45.677591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:45.678568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:45.678568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:45.678568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:45.678568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:45.678568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:45.678568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:45.678568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:45.678568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:45.678568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:45.678568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:45.678568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:45.678568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:45.678568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:45.678568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:45.678568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:45.678568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:45.678568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:45.679909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:45.679909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:45.679909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:45.679909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:45.679909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 361/724] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:45.684620 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721f6c9-5603-4915-b638-c66add4a5528 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:45.738014 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:45.791116 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721f6c9-5603-4915-b638-c66add4a5528 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:45.829738 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599235d-6d0c-4425-a810-d9da8e3f3833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:45.830441 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599235d-6d0c-4425-a810-d9da8e3f3833 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:45.830733 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599235d-6d0c-4425-a810-d9da8e3f3833 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:45.831016 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599235d-6d0c-4425-a810-d9da8e3f3833 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:45.831781 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599235d-6d0c-4425-a810-d9da8e3f3833 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:45.832099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599235d-6d0c-4425-a810-d9da8e3f3833 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:45.833569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599235d-6d0c-4425-a810-d9da8e3f3833 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:45.850351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 364/725] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:45.857999 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb13de0-a74f-48df-a47c-0336328d640a None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:45.900586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2755086-f8ae-49b9-a694-c0bfad21be19 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:45.900952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2755086-f8ae-49b9-a694-c0bfad21be19 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:45.901180 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2755086-f8ae-49b9-a694-c0bfad21be19 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:45.902598 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 365/726] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:21:45.908695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30cb30-2a5f-4e77-919d-558ac74bb12a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:45.909023 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30cb30-2a5f-4e77-919d-558ac74bb12a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:45.909285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30cb30-2a5f-4e77-919d-558ac74bb12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:45.927655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-7a30cb30-2a5f-4e77-919d-558ac74bb12a None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:45.928930 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7a30cb30-2a5f-4e77-919d-558ac74bb12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:45.986894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:45.997100 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb13de0-a74f-48df-a47c-0336328d640a None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:46.001361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 366/727] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:21:46.003428 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ff84d-9116-45cb-b174-02f3918cbfc8 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:46.003910 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ff84d-9116-45cb-b174-02f3918cbfc8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:46.004374 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ff84d-9116-45cb-b174-02f3918cbfc8 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:46.006205 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 367/728] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:46.012687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8fa05-c6fe-4a6d-840b-b891c49847b6 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:46.013358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8fa05-c6fe-4a6d-840b-b891c49847b6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:46.013881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8fa05-c6fe-4a6d-840b-b891c49847b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:46.034694 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-04f8fa05-c6fe-4a6d-840b-b891c49847b6 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:46.036268 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-04f8fa05-c6fe-4a6d-840b-b891c49847b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:46.040500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149926d-8ab4-4bf4-9495-43deae97aba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:46.040500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149926d-8ab4-4bf4-9495-43deae97aba8 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:46.040825 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149926d-8ab4-4bf4-9495-43deae97aba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:46.041073 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149926d-8ab4-4bf4-9495-43deae97aba8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:46.050984 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149926d-8ab4-4bf4-9495-43deae97aba8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:46.052372 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149926d-8ab4-4bf4-9495-43deae97aba8 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:46.068143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-d149926d-8ab4-4bf4-9495-43deae97aba8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:21:46.068143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:46.068143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:46.068143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:46.068143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:46.068143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:46.068143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:46.068143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:46.068143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:46.068143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:46.068143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:46.068143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:46.068143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:46.068143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:46.068143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:46.068143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:46.069284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:46.069284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:46.069284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:46.069284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:46.069284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:46.069284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:46.069284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:46.069284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:46.069284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:46.069284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:46.069284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:46.069284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:46.069284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:46.069284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:46.069284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:46.069284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:46.069284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:46.069284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:46.070284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:46.070284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:46.070284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:46.070284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:46.070284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:46.070284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:46.070284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:46.070284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:46.070284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:21:46.070284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:46.070284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 362/729] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:21:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:46.070940 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149926d-8ab4-4bf4-9495-43deae97aba8 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:46.097525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:46.121781 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 368/730] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:21:46.122430 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8fa05-c6fe-4a6d-840b-b891c49847b6 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:46.197815 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149926d-8ab4-4bf4-9495-43deae97aba8 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:46.237763 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca396e22-9182-4647-8f27-393fea23e82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:46.238511 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca396e22-9182-4647-8f27-393fea23e82a None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:46.239144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca396e22-9182-4647-8f27-393fea23e82a None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:46.239422 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca396e22-9182-4647-8f27-393fea23e82a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:46.244939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8fa05-c6fe-4a6d-840b-b891c49847b6 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:46.251687 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca396e22-9182-4647-8f27-393fea23e82a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:46.259606 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-ca396e22-9182-4647-8f27-393fea23e82a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:21:46.259606 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:46.259606 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:46.259606 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:46.259606 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:46.259606 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:46.259606 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:46.259606 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:46.259606 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:46.259606 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:46.259606 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:46.259606 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:46.259606 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:46.259606 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:46.259606 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:46.259606 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:46.260604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:46.260604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:46.260604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:46.260604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:46.260604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:46.260604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:46.260604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:46.260604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:46.260604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:46.260604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:46.260604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:46.260604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:46.260604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:46.260604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:46.260604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:46.260604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:46.261567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:46.261567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:46.261567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:46.261567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:46.261567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:46.261567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:46.261567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:46.261567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:46.261567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:46.261567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:46.261567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:21:46.261567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:46.261567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 363/731] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:21:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:46.262294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943de4b-0f60-4727-881a-74d539836a2e None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:46.262294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943de4b-0f60-4727-881a-74d539836a2e None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:46.263058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943de4b-0f60-4727-881a-74d539836a2e None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:46.263972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 364/732] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:46.267997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943de4b-0f60-4727-881a-74d539836a2e None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:46.282410 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:46.289887 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb96a50-fac9-4e5f-bfd0-2f4cd1b2f182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:46.290605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb96a50-fac9-4e5f-bfd0-2f4cd1b2f182 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:46.290894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb96a50-fac9-4e5f-bfd0-2f4cd1b2f182 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:46.291175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb96a50-fac9-4e5f-bfd0-2f4cd1b2f182 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:46.292019 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb96a50-fac9-4e5f-bfd0-2f4cd1b2f182 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:46.292343 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb96a50-fac9-4e5f-bfd0-2f4cd1b2f182 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:46.293844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb96a50-fac9-4e5f-bfd0-2f4cd1b2f182 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:46.317841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 369/733] 10.222.0.40 () {56 vars in 1135 bytes} [Thu Jun 16 17:21:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:46.319384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb96a50-fac9-4e5f-bfd0-2f4cd1b2f182 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:46.357908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:46.386509 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943de4b-0f60-4727-881a-74d539836a2e None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:46.424925 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a7028-abfe-4082-b35e-7c19b10d1fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:46.425982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a7028-abfe-4082-b35e-7c19b10d1fdf None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:46.426626 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a7028-abfe-4082-b35e-7c19b10d1fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:46.426999 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a7028-abfe-4082-b35e-7c19b10d1fdf None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:46.427743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a7028-abfe-4082-b35e-7c19b10d1fdf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:46.429142 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a7028-abfe-4082-b35e-7c19b10d1fdf None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:46.440807 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb96a50-fac9-4e5f-bfd0-2f4cd1b2f182 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:46.446538 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 365/734] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 17:21:46 2022] POST /identity/v3/services => generated 251 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:21:46.447992 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beea67e-70ce-4542-8be6-abda72167e45 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:46.448455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beea67e-70ce-4542-8be6-abda72167e45 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:46.448903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beea67e-70ce-4542-8be6-abda72167e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:46.474597 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-0beea67e-70ce-4542-8be6-abda72167e45 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:46.476624 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0beea67e-70ce-4542-8be6-abda72167e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:46.490620 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5520a-c644-4b92-a708-3924b4eb1540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:46.491326 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5520a-c644-4b92-a708-3924b4eb1540 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:46.491625 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5520a-c644-4b92-a708-3924b4eb1540 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:46.491916 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5520a-c644-4b92-a708-3924b4eb1540 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:46.492692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f5520a-c644-4b92-a708-3924b4eb1540 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:46.493022 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f5520a-c644-4b92-a708-3924b4eb1540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:46.494536 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f5520a-c644-4b92-a708-3924b4eb1540 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:46.510256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 370/735] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:46.510256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5520a-c644-4b92-a708-3924b4eb1540 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:46.575819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:46.589508 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 366/736] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:46.590839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beea67e-70ce-4542-8be6-abda72167e45 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:46.631300 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:46.650385 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5520a-c644-4b92-a708-3924b4eb1540 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:46.707129 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a035444-74d0-484c-91be-811574651512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:46.707806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a035444-74d0-484c-91be-811574651512 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:46.708094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a035444-74d0-484c-91be-811574651512 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:46.708373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a035444-74d0-484c-91be-811574651512 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:46.744600 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beea67e-70ce-4542-8be6-abda72167e45 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:46.787292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a035444-74d0-484c-91be-811574651512 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:46.788763 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a035444-74d0-484c-91be-811574651512 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:46.789682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385cc61-4353-4185-9c08-a54fd172c42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:46.790828 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385cc61-4353-4185-9c08-a54fd172c42c None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:46.791190 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385cc61-4353-4185-9c08-a54fd172c42c None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:46.792379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385cc61-4353-4185-9c08-a54fd172c42c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:46.806592 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385cc61-4353-4185-9c08-a54fd172c42c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:46.808673 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385cc61-4353-4185-9c08-a54fd172c42c None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:46.831276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-f385cc61-4353-4185-9c08-a54fd172c42c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:46.831276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:46.831276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:46.831276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:46.831276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:46.831276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:46.831276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:46.831276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:46.831276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:46.831276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:46.831276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:46.831276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:46.831276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:46.831276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:46.831276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:46.832307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:46.832307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:46.832307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:46.832307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:46.832307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:46.832307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:46.832307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:46.832307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:46.832307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:46.832307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:46.832307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:46.832307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:46.832307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:46.832307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:46.832307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:46.832307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:46.832307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:46.833267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:46.833267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:46.833267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:46.833267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:46.833267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 367/737] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:46.834002 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385cc61-4353-4185-9c08-a54fd172c42c None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:46.899518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:46.923092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 371/738] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:46 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:21:46.923582 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:46.924357 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a035444-74d0-484c-91be-811574651512 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:46.946028 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385cc61-4353-4185-9c08-a54fd172c42c None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:46.985177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05439f-a478-443d-94cf-1b2906a20e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:46.986698 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05439f-a478-443d-94cf-1b2906a20e84 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:46.987150 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05439f-a478-443d-94cf-1b2906a20e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:46.987574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05439f-a478-443d-94cf-1b2906a20e84 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:47.002451 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05439f-a478-443d-94cf-1b2906a20e84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:47.013310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-6d05439f-a478-443d-94cf-1b2906a20e84 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:47.013310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:47.013310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:47.013310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:47.013310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:47.013310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:47.013310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:47.013310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:47.013310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:47.013310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:47.013310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:47.013310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:47.013310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:47.013310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:47.013310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:47.013310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:47.014501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:47.014501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:47.014501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:47.014501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:47.014501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:47.014501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:47.014501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:47.014501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:47.014501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:47.014501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:47.014501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:47.014501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:47.014501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:47.014501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:47.014501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:47.014501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:47.016273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:47.016273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:47.016273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:47.016273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:47.016273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:47.016273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:47.016273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:47.016273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:47.016273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:47.016273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:47.016273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:47.016273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:47.020118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 368/739] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:21:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:47.021256 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05439f-a478-443d-94cf-1b2906a20e84 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:47.058377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a035444-74d0-484c-91be-811574651512 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:47.105100 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6451db8f-a011-4df4-b7d9-5d442e02d8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:47.106057 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6451db8f-a011-4df4-b7d9-5d442e02d8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:47.106338 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6451db8f-a011-4df4-b7d9-5d442e02d8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:47.106620 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6451db8f-a011-4df4-b7d9-5d442e02d8f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:47.107349 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6451db8f-a011-4df4-b7d9-5d442e02d8f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:47.107659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6451db8f-a011-4df4-b7d9-5d442e02d8f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:47.109047 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6451db8f-a011-4df4-b7d9-5d442e02d8f1 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:47.120553 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 372/740] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:47.121384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6451db8f-a011-4df4-b7d9-5d442e02d8f1 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:47.147620 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05439f-a478-443d-94cf-1b2906a20e84 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:47.164136 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:47.176549 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:47.188665 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3731e94-079b-4528-a796-1e6000283b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:47.189600 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3731e94-079b-4528-a796-1e6000283b52 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:47.189887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3731e94-079b-4528-a796-1e6000283b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:47.190278 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3731e94-079b-4528-a796-1e6000283b52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:47.202325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3731e94-079b-4528-a796-1e6000283b52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:47.204952 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3731e94-079b-4528-a796-1e6000283b52 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:47.223748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-a3731e94-079b-4528-a796-1e6000283b52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:47.223748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:47.223748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:47.223748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:47.223748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:47.223748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:47.223748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:47.223748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:47.223748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:47.223748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:47.223748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:47.223748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:47.223748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:47.223748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:47.223748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:47.224697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:47.224697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:47.224697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:47.224697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:47.224697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:47.224697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:47.224697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:47.224697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:47.224697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:47.224697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:47.224697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:47.224697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:47.224697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:47.224697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:47.224697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:47.224697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:47.224697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:47.225624 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:47.225624 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:47.225624 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:47.225624 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:47.225624 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 369/741] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:47.226222 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578bb73-49ee-41ba-a820-2bfef4e30754 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:47.226521 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578bb73-49ee-41ba-a820-2bfef4e30754 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:47.226811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578bb73-49ee-41ba-a820-2bfef4e30754 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:47.228238 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 370/742] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:47.228775 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578bb73-49ee-41ba-a820-2bfef4e30754 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:47.244447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6451db8f-a011-4df4-b7d9-5d442e02d8f1 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:47.283921 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad78217-b5a7-4b62-b1f3-ceb399d1fa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:47.284608 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad78217-b5a7-4b62-b1f3-ceb399d1fa43 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:47.284899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad78217-b5a7-4b62-b1f3-ceb399d1fa43 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:47.285221 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad78217-b5a7-4b62-b1f3-ceb399d1fa43 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:47.285989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad78217-b5a7-4b62-b1f3-ceb399d1fa43 None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:47.286305 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad78217-b5a7-4b62-b1f3-ceb399d1fa43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:47.287689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad78217-b5a7-4b62-b1f3-ceb399d1fa43 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:47.300199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 373/743] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 17:21:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:47.300199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad78217-b5a7-4b62-b1f3-ceb399d1fa43 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:47.357674 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578bb73-49ee-41ba-a820-2bfef4e30754 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:47.396805 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d04af4-843a-4776-a2d9-b3bbd8dafdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:47.398029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d04af4-843a-4776-a2d9-b3bbd8dafdd8 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:47.398518 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d04af4-843a-4776-a2d9-b3bbd8dafdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:47.399008 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d04af4-843a-4776-a2d9-b3bbd8dafdd8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:47.407492 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d04af4-843a-4776-a2d9-b3bbd8dafdd8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:47.409183 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d04af4-843a-4776-a2d9-b3bbd8dafdd8 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:47.410480 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad78217-b5a7-4b62-b1f3-ceb399d1fa43 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:47.420692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:47.423568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-51d04af4-843a-4776-a2d9-b3bbd8dafdd8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:21:47.423568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:47.423568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:47.423568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:47.423568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:47.423568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:47.423568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:47.423568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:47.423568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:47.423568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:47.423568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:47.423568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:47.423568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:47.423568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:47.423568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:47.423568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:47.424569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:47.424569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:47.424569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:47.424569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:47.424569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:47.424569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:47.424569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:47.424569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:47.424569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:47.424569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:47.424569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:47.424569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:47.424569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:47.424569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:47.424569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:47.424569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:47.424569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:47.424569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:47.425594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:47.425594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:47.425594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:47.425594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:47.425594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:47.425594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:47.425594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:47.425594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:47.425594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:21:47.425594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:47.425594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 371/744] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jun 16 17:21:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:47.426729 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d9ef5-bd18-4c6e-a2f8-176595b7b742 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:47.426729 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d9ef5-bd18-4c6e-a2f8-176595b7b742 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:47.426729 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d9ef5-bd18-4c6e-a2f8-176595b7b742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:47.444568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-935d9ef5-bd18-4c6e-a2f8-176595b7b742 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:47.445791 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-935d9ef5-bd18-4c6e-a2f8-176595b7b742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:47.456099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35a284-7e61-4478-b9d6-6fc60a9578f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:47.456778 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35a284-7e61-4478-b9d6-6fc60a9578f3 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:47.457069 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35a284-7e61-4478-b9d6-6fc60a9578f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:47.457385 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35a284-7e61-4478-b9d6-6fc60a9578f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:47.458132 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35a284-7e61-4478-b9d6-6fc60a9578f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:47.458434 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35a284-7e61-4478-b9d6-6fc60a9578f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:47.459813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35a284-7e61-4478-b9d6-6fc60a9578f3 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:47.465993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:47.473694 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 374/745] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:47.474498 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35a284-7e61-4478-b9d6-6fc60a9578f3 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:47.529188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 372/746] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:21:47.529188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d9ef5-bd18-4c6e-a2f8-176595b7b742 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:47.602699 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35a284-7e61-4478-b9d6-6fc60a9578f3 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:47.644270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d9ef5-bd18-4c6e-a2f8-176595b7b742 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:47.650281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f6465-fd35-45dc-b972-d3b2abff91ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:47.651094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f6465-fd35-45dc-b972-d3b2abff91ca None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:47.651384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f6465-fd35-45dc-b972-d3b2abff91ca None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:47.652084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f6465-fd35-45dc-b972-d3b2abff91ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:47.663316 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f6465-fd35-45dc-b972-d3b2abff91ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:47.664781 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f6465-fd35-45dc-b972-d3b2abff91ca None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:47.686323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-360f6465-fd35-45dc-b972-d3b2abff91ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:47.686323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:47.686323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:47.686323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:47.686323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:47.686323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:47.686323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:47.686323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:47.686323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:47.686323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:47.686323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:47.686323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:47.686323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:47.686323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:47.686323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:47.688263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:47.688263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:47.688263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:47.688263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:47.688263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:47.688263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:47.688263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:47.688263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:47.688263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:47.688263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:47.688263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:47.688263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:47.688263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:47.688263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:47.688263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:47.688263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:47.688263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:47.689253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:47.689253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:47.689253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:47.689253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:47.689482 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59622e29-2a60-4da3-b18f-803e7fe4a8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:47.689482 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59622e29-2a60-4da3-b18f-803e7fe4a8bb None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:47.689629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 375/747] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:47.689629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f6465-fd35-45dc-b972-d3b2abff91ca None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:47.692936 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59622e29-2a60-4da3-b18f-803e7fe4a8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:47.693214 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59622e29-2a60-4da3-b18f-803e7fe4a8bb None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:47.694125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59622e29-2a60-4da3-b18f-803e7fe4a8bb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:47.694451 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59622e29-2a60-4da3-b18f-803e7fe4a8bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:47.695916 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59622e29-2a60-4da3-b18f-803e7fe4a8bb None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:47.713253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:47.715581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 373/748] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jun 16 17:21:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:47.716638 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59622e29-2a60-4da3-b18f-803e7fe4a8bb None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:47.735594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:47.804536 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f6465-fd35-45dc-b972-d3b2abff91ca None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:47.829944 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59622e29-2a60-4da3-b18f-803e7fe4a8bb None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:47.841777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70572e73-d2f7-4819-8361-4334c13933b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:47.842586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70572e73-d2f7-4819-8361-4334c13933b0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:47.842873 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70572e73-d2f7-4819-8361-4334c13933b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:47.843151 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70572e73-d2f7-4819-8361-4334c13933b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:47.850231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70572e73-d2f7-4819-8361-4334c13933b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:47.851685 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70572e73-d2f7-4819-8361-4334c13933b0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:47.866098 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e1fac-4701-4a03-956a-08d82c9b2bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:47.866681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-70572e73-d2f7-4819-8361-4334c13933b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:47.866681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:47.866681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:47.866681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:47.866681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:47.866681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:47.866681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:47.866681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:47.866681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:47.866681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:47.866681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:47.866681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:47.866681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:47.866681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:47.866681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:47.867770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:47.867770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:47.867770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:47.867770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:47.867770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:47.867770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:47.867770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:47.867770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:47.867770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:47.867770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:47.867770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:47.867770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:47.867770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:47.867770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:47.867770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:47.867770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:47.867770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:47.868896 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:47.868896 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:47.868896 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:47.868896 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:47.868896 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:47.868896 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:47.868896 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:47.868896 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:47.869463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e1fac-4701-4a03-956a-08d82c9b2bd3 None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:47.869463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e1fac-4701-4a03-956a-08d82c9b2bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:47.869463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e1fac-4701-4a03-956a-08d82c9b2bd3 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:47.869463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e1fac-4701-4a03-956a-08d82c9b2bd3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:47.869776 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e1fac-4701-4a03-956a-08d82c9b2bd3 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:47.876287 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 376/749] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:47.876287 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70572e73-d2f7-4819-8361-4334c13933b0 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:47.893434 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 374/750] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 17:21:47 2022] POST /identity/v3/services => generated 240 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:21:47.893434 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e1fac-4701-4a03-956a-08d82c9b2bd3 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:48.002279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:48.003128 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:48.028565 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70572e73-d2f7-4819-8361-4334c13933b0 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:48.061819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e1fac-4701-4a03-956a-08d82c9b2bd3 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:48.070600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23170dd4-298d-4def-a151-d0166d1b2447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:48.071392 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23170dd4-298d-4def-a151-d0166d1b2447 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:48.071758 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23170dd4-298d-4def-a151-d0166d1b2447 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:48.072104 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23170dd4-298d-4def-a151-d0166d1b2447 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:48.072921 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23170dd4-298d-4def-a151-d0166d1b2447 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:48.073443 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23170dd4-298d-4def-a151-d0166d1b2447 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:48.075174 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23170dd4-298d-4def-a151-d0166d1b2447 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:48.090162 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 377/751] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:48.091092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23170dd4-298d-4def-a151-d0166d1b2447 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:48.110077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a96520-ac22-4aad-ac93-10a7a4409843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:48.111348 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a96520-ac22-4aad-ac93-10a7a4409843 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:48.111647 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a96520-ac22-4aad-ac93-10a7a4409843 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:48.111937 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a96520-ac22-4aad-ac93-10a7a4409843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:48.125542 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a96520-ac22-4aad-ac93-10a7a4409843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:48.127038 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a96520-ac22-4aad-ac93-10a7a4409843 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:48.150063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-72a96520-ac22-4aad-ac93-10a7a4409843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:48.150063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:48.150063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:48.150063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:48.150063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:48.150063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:48.150063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:48.150063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:48.150063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:48.150063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:48.150063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:48.150063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:48.150063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:48.150063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:48.150063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:48.151049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:48.151049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:48.151049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:48.151049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:48.151049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:48.151049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:48.151049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:48.151049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:48.151049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:48.151049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:48.151049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:48.151049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:48.151049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:48.151049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:48.151049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:48.151049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:48.151049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:48.152015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:48.152015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:48.152015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:48.152015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:48.152015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 375/752] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:48.152298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a96520-ac22-4aad-ac93-10a7a4409843 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:48.209182 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23170dd4-298d-4def-a151-d0166d1b2447 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:48.258708 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aee58f-2b76-4c7b-8193-99fadf2fdb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:48.259397 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aee58f-2b76-4c7b-8193-99fadf2fdb0a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:48.259741 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aee58f-2b76-4c7b-8193-99fadf2fdb0a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:48.260103 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aee58f-2b76-4c7b-8193-99fadf2fdb0a None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:48.260969 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aee58f-2b76-4c7b-8193-99fadf2fdb0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:48.261364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aee58f-2b76-4c7b-8193-99fadf2fdb0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:48.262878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aee58f-2b76-4c7b-8193-99fadf2fdb0a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:48.274082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:48.277950 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 378/753] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:48.284017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aee58f-2b76-4c7b-8193-99fadf2fdb0a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:48.317134 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a96520-ac22-4aad-ac93-10a7a4409843 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:48.342069 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:48.361091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f9a12-f1c8-42a2-afa4-17f8aee21e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:48.363111 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f9a12-f1c8-42a2-afa4-17f8aee21e71 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:48.363581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f9a12-f1c8-42a2-afa4-17f8aee21e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:48.364299 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f9a12-f1c8-42a2-afa4-17f8aee21e71 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:48.377902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f9a12-f1c8-42a2-afa4-17f8aee21e71 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:48.380224 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f9a12-f1c8-42a2-afa4-17f8aee21e71 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:48.403201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-570f9a12-f1c8-42a2-afa4-17f8aee21e71 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:21:48.403201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:48.403201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:48.403201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:48.403201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:48.403201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:48.403201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:48.403201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:48.403201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:48.403201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:48.403201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:48.403201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:48.403201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:48.403201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:48.403201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:48.403201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:48.404463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:48.404463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:48.404463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:48.404463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:48.404463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:48.404463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:48.404463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:48.404463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:48.404463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:48.404463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:48.404463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:48.404463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:48.404463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:48.404463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:48.404463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:48.404463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:48.404463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:48.404463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:48.405735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:48.405735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:48.405735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:48.405735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:48.405735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:48.405735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:48.405735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:48.405735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:48.405735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:21:48.405735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:48.409666 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 376/754] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jun 16 17:21:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:48.411025 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f9a12-f1c8-42a2-afa4-17f8aee21e71 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:48.414841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aee58f-2b76-4c7b-8193-99fadf2fdb0a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:48.451245 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbf19f-dec0-482d-8e08-ac1cbecad03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:48.452130 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbf19f-dec0-482d-8e08-ac1cbecad03c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:48.452539 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbf19f-dec0-482d-8e08-ac1cbecad03c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:48.453494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbf19f-dec0-482d-8e08-ac1cbecad03c None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:48.454322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cbf19f-dec0-482d-8e08-ac1cbecad03c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:48.454744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cbf19f-dec0-482d-8e08-ac1cbecad03c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:48.456209 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cbf19f-dec0-482d-8e08-ac1cbecad03c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:48.469777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 379/755] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:48.470891 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbf19f-dec0-482d-8e08-ac1cbecad03c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:48.535989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:48.550297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f9a12-f1c8-42a2-afa4-17f8aee21e71 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:48.579755 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbf19f-dec0-482d-8e08-ac1cbecad03c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:48.587644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d11b01-9a64-4eb2-8fd9-a678559dc6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:48.588495 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d11b01-9a64-4eb2-8fd9-a678559dc6dc None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:48.588936 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d11b01-9a64-4eb2-8fd9-a678559dc6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:48.589408 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d11b01-9a64-4eb2-8fd9-a678559dc6dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:48.590564 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d11b01-9a64-4eb2-8fd9-a678559dc6dc None admin] RBAC: Adding query filter params (role.id=25a16d8ea4084e9eacdebb7a4dcd2337, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=b8a37d1201d54afdad9b228ef4f2d787) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:48.591166 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d11b01-9a64-4eb2-8fd9-a678559dc6dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:48.592820 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d11b01-9a64-4eb2-8fd9-a678559dc6dc None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:48.605617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 377/756] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:21:48 2022] GET /identity/v3/role_assignments?user.id=b8a37d1201d54afdad9b228ef4f2d787&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=25a16d8ea4084e9eacdebb7a4dcd2337 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:48.606964 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d11b01-9a64-4eb2-8fd9-a678559dc6dc None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:48.617810 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e9b42-7a8f-45f4-86a0-ef4eba78eb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:48.618489 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e9b42-7a8f-45f4-86a0-ef4eba78eb39 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:48.618768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e9b42-7a8f-45f4-86a0-ef4eba78eb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:48.619045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e9b42-7a8f-45f4-86a0-ef4eba78eb39 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:48.619894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e9b42-7a8f-45f4-86a0-ef4eba78eb39 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:48.620196 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e9b42-7a8f-45f4-86a0-ef4eba78eb39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:48.621653 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e9b42-7a8f-45f4-86a0-ef4eba78eb39 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:48.645200 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 380/757] 10.222.0.40 () {56 vars in 1149 bytes} [Thu Jun 16 17:21:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:48.647788 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e9b42-7a8f-45f4-86a0-ef4eba78eb39 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:48.700173 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:48.803701 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d11b01-9a64-4eb2-8fd9-a678559dc6dc None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:48.829809 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e9b42-7a8f-45f4-86a0-ef4eba78eb39 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:48.842102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd35813-9870-471c-97ce-fa15710b9414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:48.843320 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd35813-9870-471c-97ce-fa15710b9414 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:48.844318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd35813-9870-471c-97ce-fa15710b9414 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:48.844962 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd35813-9870-471c-97ce-fa15710b9414 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:48.850327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:48.853801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd35813-9870-471c-97ce-fa15710b9414 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:48.855446 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd35813-9870-471c-97ce-fa15710b9414 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:48.869015 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1367bc5-e627-4258-8113-0386a15295d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:48.869698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1367bc5-e627-4258-8113-0386a15295d1 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:48.870030 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1367bc5-e627-4258-8113-0386a15295d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:48.870315 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1367bc5-e627-4258-8113-0386a15295d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:48.873400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-0dd35813-9870-471c-97ce-fa15710b9414 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:48.873400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:48.873400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:48.873400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:48.873400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:48.873400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:48.873400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:48.873400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:48.873400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:48.873400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:48.873400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:48.873400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:48.873400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:48.873400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:48.873400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:48.874309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:48.874309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:48.874309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:48.874309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:48.874309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:48.874309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:48.874309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:48.874309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:48.874309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:48.874309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:48.874309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:48.874309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:48.874309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:48.874309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:48.874309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:48.874309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:48.874309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:48.875277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:48.875277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:48.875277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:48.875277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:48.875277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:48.875277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:48.875277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:48.875277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:48.877848 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 378/758] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:48.879584 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce6036-26e7-4810-a44f-242ed90c71c4 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:48.880080 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce6036-26e7-4810-a44f-242ed90c71c4 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:48.880544 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce6036-26e7-4810-a44f-242ed90c71c4 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:48.886879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 379/759] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:48.887656 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce6036-26e7-4810-a44f-242ed90c71c4 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:48.955390 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1367bc5-e627-4258-8113-0386a15295d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:48.956858 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1367bc5-e627-4258-8113-0386a15295d1 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:49.011988 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce6036-26e7-4810-a44f-242ed90c71c4 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:49.015876 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:49.051409 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21936559-5b73-4c92-aeec-d6f9e215300f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:49.051786 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21936559-5b73-4c92-aeec-d6f9e215300f None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:49.052087 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21936559-5b73-4c92-aeec-d6f9e215300f None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:49.052371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21936559-5b73-4c92-aeec-d6f9e215300f None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:49.053197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21936559-5b73-4c92-aeec-d6f9e215300f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:49.053535 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21936559-5b73-4c92-aeec-d6f9e215300f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:49.055034 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21936559-5b73-4c92-aeec-d6f9e215300f None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:49.070239 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 380/760] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:49.071714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e4921-8b7b-4e0b-8218-1eb273771b07 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:49.072003 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e4921-8b7b-4e0b-8218-1eb273771b07 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:49.072352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e4921-8b7b-4e0b-8218-1eb273771b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:49.090494 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-018e4921-8b7b-4e0b-8218-1eb273771b07 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:49.091750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-018e4921-8b7b-4e0b-8218-1eb273771b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:49.094538 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 381/761] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:21:49.096825 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1367bc5-e627-4258-8113-0386a15295d1 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:49.121903 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:49.182974 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 381/762] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:21:49.184174 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e4921-8b7b-4e0b-8218-1eb273771b07 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:49.252308 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1367bc5-e627-4258-8113-0386a15295d1 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:49.289272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3c6aa-96e0-4c35-80fd-8be4ad23c368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:49.289840 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3c6aa-96e0-4c35-80fd-8be4ad23c368 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:49.290129 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3c6aa-96e0-4c35-80fd-8be4ad23c368 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:49.290412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3c6aa-96e0-4c35-80fd-8be4ad23c368 None admin] PATH_INFO: `/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/689a31e26c6b454a91c3fd79678b4d07/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:49.292457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f3c6aa-96e0-4c35-80fd-8be4ad23c368 None admin] RBAC: Authorizing `identity:create_grant(project_id=35a535a305e740368b7a4d77d0ad22a0, user_id=689a31e26c6b454a91c3fd79678b4d07, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:49.294002 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f3c6aa-96e0-4c35-80fd-8be4ad23c368 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:49.310365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 382/763] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:21:49 2022] PUT /identity/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/689a31e26c6b454a91c3fd79678b4d07/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:21:49.311285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3c6aa-96e0-4c35-80fd-8be4ad23c368 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:49.320669 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:49.351031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e4921-8b7b-4e0b-8218-1eb273771b07 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:49.442085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60f0ff-d524-4300-aee7-3f3f762c3b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:49.443787 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60f0ff-d524-4300-aee7-3f3f762c3b6e None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:49.444306 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60f0ff-d524-4300-aee7-3f3f762c3b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:49.444799 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60f0ff-d524-4300-aee7-3f3f762c3b6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:49.471189 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e60f0ff-d524-4300-aee7-3f3f762c3b6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:49.473031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e60f0ff-d524-4300-aee7-3f3f762c3b6e None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:49.489512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:49.507119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-1e60f0ff-d524-4300-aee7-3f3f762c3b6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:49.507119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:49.507119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:49.507119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:49.507119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:49.507119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:49.507119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:49.507119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:49.507119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:49.507119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:49.507119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:49.507119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:49.507119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:49.507119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:49.507119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:49.508367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:49.508367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:49.508367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:49.508367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:49.508367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:49.508367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:49.508367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:49.508367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:49.508367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:49.508367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:49.508367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:49.508367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:49.508367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:49.508367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:49.508367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:49.508367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:49.508367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:49.509321 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:49.509321 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:49.509321 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:49.509321 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:49.511724 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 382/764] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:49.516165 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60f0ff-d524-4300-aee7-3f3f762c3b6e None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:49.517289 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3c6aa-96e0-4c35-80fd-8be4ad23c368 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:49.568184 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af11e7e-fbde-40fc-817d-cc46131e9b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:49.570228 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af11e7e-fbde-40fc-817d-cc46131e9b61 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:49.570728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af11e7e-fbde-40fc-817d-cc46131e9b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:49.571214 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af11e7e-fbde-40fc-817d-cc46131e9b61 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:49.572098 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af11e7e-fbde-40fc-817d-cc46131e9b61 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:49.573720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af11e7e-fbde-40fc-817d-cc46131e9b61 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:49.592033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-7af11e7e-fbde-40fc-817d-cc46131e9b61 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:21:49.592033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:49.592033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:21:49.592033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:21:49.592033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:21:49.592033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:21:49.592033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:21:49.592033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:21:49.592033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:21:49.592033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:49.592033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:21:49.592033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:49.592033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:49.592033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:49.592033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:49.592033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:49.593041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:49.593041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:49.593041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:49.593041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:49.593041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:49.593041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:49.593041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:49.593041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:21:49.593041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:21:49.593041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:21:49.593041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:21:49.593041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:49.593041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:49.593041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:49.593041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:49.593041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:49.594769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:49.594769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:49.594769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:49.594769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:49.594769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:49.594769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:49.594769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:49.594769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:49.594769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:49.594769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:49.594769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:49.594769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:21:49.594769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:21:49.594769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:21:49.594769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:49.594769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 383/765] 10.222.0.40 () {56 vars in 1129 bytes} [Thu Jun 16 17:21:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:49.601047 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af11e7e-fbde-40fc-817d-cc46131e9b61 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:49.682150 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60f0ff-d524-4300-aee7-3f3f762c3b6e None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:49.755834 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af11e7e-fbde-40fc-817d-cc46131e9b61 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:49.776828 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543cec5-46a0-4832-b77f-f15a58df6c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:49.777615 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543cec5-46a0-4832-b77f-f15a58df6c6b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:49.777917 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543cec5-46a0-4832-b77f-f15a58df6c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:49.778207 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543cec5-46a0-4832-b77f-f15a58df6c6b None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:49.778953 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543cec5-46a0-4832-b77f-f15a58df6c6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:49.779271 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543cec5-46a0-4832-b77f-f15a58df6c6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:49.780651 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543cec5-46a0-4832-b77f-f15a58df6c6b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:49.800248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9114e5-d19a-4fb4-8fb3-b5dce62ed319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:49.801218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9114e5-d19a-4fb4-8fb3-b5dce62ed319 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:49.801658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9114e5-d19a-4fb4-8fb3-b5dce62ed319 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:49.802088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9114e5-d19a-4fb4-8fb3-b5dce62ed319 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:49.802961 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9114e5-d19a-4fb4-8fb3-b5dce62ed319 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:49.803525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9114e5-d19a-4fb4-8fb3-b5dce62ed319 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:49.807228 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9114e5-d19a-4fb4-8fb3-b5dce62ed319 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:49.813374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 383/766] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:49.816763 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 384/767] 10.222.0.40 () {56 vars in 1124 bytes} [Thu Jun 16 17:21:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:49.826432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543cec5-46a0-4832-b77f-f15a58df6c6b None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:49.826620 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9114e5-d19a-4fb4-8fb3-b5dce62ed319 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:49.849866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:49.850405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:49.973565 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543cec5-46a0-4832-b77f-f15a58df6c6b None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:50.017864 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db9ba4-487c-4b87-91ca-24381983766e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:50.018944 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db9ba4-487c-4b87-91ca-24381983766e None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:50.019434 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db9ba4-487c-4b87-91ca-24381983766e None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:50.019721 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db9ba4-487c-4b87-91ca-24381983766e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:50.038053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db9ba4-487c-4b87-91ca-24381983766e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:50.039787 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db9ba4-487c-4b87-91ca-24381983766e None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:50.049699 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9114e5-d19a-4fb4-8fb3-b5dce62ed319 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:50.064946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-56db9ba4-487c-4b87-91ca-24381983766e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:50.064946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:50.064946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:50.064946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:50.064946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:50.064946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:50.064946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:50.064946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:50.064946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:50.064946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:50.064946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:50.064946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:50.064946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:50.064946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:50.064946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:50.065926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:50.065926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:50.065926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:50.065926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:50.065926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:50.065926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:50.065926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:50.065926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:50.065926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:50.065926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:50.065926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:50.065926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:50.065926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:50.065926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:50.065926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:50.065926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:50.065926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:50.066844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:50.066844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:50.066844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:50.066844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:50.066844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:50.066844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:50.066844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:50.066844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:50.070425 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 384/768] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:50.079187 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db9ba4-487c-4b87-91ca-24381983766e None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:50.116322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefb73d-2d97-4ce0-8cc4-68e867aa9a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:50.118571 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefb73d-2d97-4ce0-8cc4-68e867aa9a64 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:50.119024 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefb73d-2d97-4ce0-8cc4-68e867aa9a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:50.119463 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefb73d-2d97-4ce0-8cc4-68e867aa9a64 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:50.120495 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcefb73d-2d97-4ce0-8cc4-68e867aa9a64 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:50.121722 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcefb73d-2d97-4ce0-8cc4-68e867aa9a64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:50.123871 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcefb73d-2d97-4ce0-8cc4-68e867aa9a64 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:50.134032 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 385/769] 10.222.0.40 () {56 vars in 1124 bytes} [Thu Jun 16 17:21:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:50.141685 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefb73d-2d97-4ce0-8cc4-68e867aa9a64 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:50.197565 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:50.244082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:50.254368 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db9ba4-487c-4b87-91ca-24381983766e None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:50.310606 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefb73d-2d97-4ce0-8cc4-68e867aa9a64 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:50.333554 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45a26c-3e81-4398-b3fd-9fff735ae3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:50.334165 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45a26c-3e81-4398-b3fd-9fff735ae3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:50.334457 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45a26c-3e81-4398-b3fd-9fff735ae3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:50.334740 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45a26c-3e81-4398-b3fd-9fff735ae3e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:50.335563 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45a26c-3e81-4398-b3fd-9fff735ae3e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:50.335886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45a26c-3e81-4398-b3fd-9fff735ae3e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:50.337441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45a26c-3e81-4398-b3fd-9fff735ae3e7 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:50.363758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 385/770] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:50.367225 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150b577-a853-4dfc-9c08-958fa64c2915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:50.367923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150b577-a853-4dfc-9c08-958fa64c2915 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:50.368478 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150b577-a853-4dfc-9c08-958fa64c2915 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:50.368767 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150b577-a853-4dfc-9c08-958fa64c2915 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:50.369140 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45a26c-3e81-4398-b3fd-9fff735ae3e7 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:50.369638 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150b577-a853-4dfc-9c08-958fa64c2915 None admin] RBAC: Adding query filter params (interface=public, service_id=3ef2469270f24118981c79bee12e8958, region_id=RegionOne) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:50.370065 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150b577-a853-4dfc-9c08-958fa64c2915 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:50.371469 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150b577-a853-4dfc-9c08-958fa64c2915 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:50.392015 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 386/771] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jun 16 17:21:50 2022] GET /identity/v3/endpoints?service_id=3ef2469270f24118981c79bee12e8958&interface=public®ion_id=RegionOne => generated 194 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:50.396783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150b577-a853-4dfc-9c08-958fa64c2915 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:50.502962 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45a26c-3e81-4398-b3fd-9fff735ae3e7 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:50.509257 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150b577-a853-4dfc-9c08-958fa64c2915 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:50.529425 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:50.539510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d355a8-c4da-4800-908d-fa9ce02213cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:50.540179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d355a8-c4da-4800-908d-fa9ce02213cc None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:50.540774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d355a8-c4da-4800-908d-fa9ce02213cc None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:50.541057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d355a8-c4da-4800-908d-fa9ce02213cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:50.542250 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d355a8-c4da-4800-908d-fa9ce02213cc None admin] RBAC: Adding query filter params (role.id=0e34a6f9b89a4e29951fb189abacb34a, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=cfc2f31f654e486d8505d80aad17d17c) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:50.542613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d355a8-c4da-4800-908d-fa9ce02213cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:50.544504 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d355a8-c4da-4800-908d-fa9ce02213cc None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:50.549271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505dc556-1209-43d2-8f89-c80f60805bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:50.550216 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dc556-1209-43d2-8f89-c80f60805bad None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:50.557179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dc556-1209-43d2-8f89-c80f60805bad None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:50.557367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 386/772] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:21:50 2022] GET /identity/v3/role_assignments?user.id=cfc2f31f654e486d8505d80aad17d17c&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=0e34a6f9b89a4e29951fb189abacb34a => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:50.557367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800668fa-97fd-4616-9627-e7d0d19315aa None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:50.557367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800668fa-97fd-4616-9627-e7d0d19315aa None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:50.557367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800668fa-97fd-4616-9627-e7d0d19315aa None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:50.557367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 387/773] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:50.557868 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dc556-1209-43d2-8f89-c80f60805bad None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:50.558502 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505dc556-1209-43d2-8f89-c80f60805bad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:50.559915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505dc556-1209-43d2-8f89-c80f60805bad None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:50.567274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 387/774] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jun 16 17:21:50 2022] GET /identity/v3/services => generated 772 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:50.567587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:50.574999 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916f2eb-e9ac-40ca-b735-ff9c1d9609a7 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:50.575648 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916f2eb-e9ac-40ca-b735-ff9c1d9609a7 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:50.576112 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916f2eb-e9ac-40ca-b735-ff9c1d9609a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:50.595703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-9916f2eb-e9ac-40ca-b735-ff9c1d9609a7 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:50.597262 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-9916f2eb-e9ac-40ca-b735-ff9c1d9609a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:50.677665 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 388/775] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:21:50.704850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505dc556-1209-43d2-8f89-c80f60805bad None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:50.880804 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505dc556-1209-43d2-8f89-c80f60805bad None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:50.922631 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b20cc2-e45f-4602-8f3a-158d15db2de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:50.923652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b20cc2-e45f-4602-8f3a-158d15db2de2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:50.923946 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b20cc2-e45f-4602-8f3a-158d15db2de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:50.924223 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b20cc2-e45f-4602-8f3a-158d15db2de2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:50.924890 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b20cc2-e45f-4602-8f3a-158d15db2de2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:50.926298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b20cc2-e45f-4602-8f3a-158d15db2de2 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:50.940265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-f2b20cc2-e45f-4602-8f3a-158d15db2de2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:21:50.940265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:50.940265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:21:50.940265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:21:50.940265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:21:50.940265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:21:50.940265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:21:50.940265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:21:50.940265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:21:50.940265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:50.940265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:21:50.940265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:50.940265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:50.940265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:50.940265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:50.940265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:50.940265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:50.941351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:50.941351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:50.941351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:50.941351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:50.941351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:50.941351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:50.941351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:21:50.941351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:21:50.941351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:21:50.941351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:21:50.941351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:50.941351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:50.941351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:50.941351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:50.941351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:50.941351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:50.942253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:50.942253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:50.942253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:50.942253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:50.942253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:50.942253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:50.942253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:50.942253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:50.942253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:50.942253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:50.942253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:21:50.942253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:21:50.942253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:21:50.942253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:50.942253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 388/776] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:50 2022] GET /identity/v3/services/network => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:50.951280 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916f2eb-e9ac-40ca-b735-ff9c1d9609a7 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:51.055881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673fbe3-77dc-4616-81fa-75ff496dc255 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:51.056226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673fbe3-77dc-4616-81fa-75ff496dc255 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:51.056445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673fbe3-77dc-4616-81fa-75ff496dc255 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:51.058127 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 389/777] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:21:51.063637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40916c4c-70b6-43bc-98c9-18e97fd3f749 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:51.063937 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40916c4c-70b6-43bc-98c9-18e97fd3f749 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:51.064249 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40916c4c-70b6-43bc-98c9-18e97fd3f749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:51.072551 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916f2eb-e9ac-40ca-b735-ff9c1d9609a7 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:51.100058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-40916c4c-70b6-43bc-98c9-18e97fd3f749 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:51.103833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-40916c4c-70b6-43bc-98c9-18e97fd3f749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:51.107174 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:51.129112 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90d05f-0a27-4029-bb2d-293d6f337343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:51.129929 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90d05f-0a27-4029-bb2d-293d6f337343 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:51.130217 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90d05f-0a27-4029-bb2d-293d6f337343 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:51.130501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90d05f-0a27-4029-bb2d-293d6f337343 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:51.131285 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90d05f-0a27-4029-bb2d-293d6f337343 None admin] RBAC: Adding query filter params (name=network) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:51.131542 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90d05f-0a27-4029-bb2d-293d6f337343 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:51.132944 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90d05f-0a27-4029-bb2d-293d6f337343 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:51.138713 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 389/778] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 17:21:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:51.145984 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90d05f-0a27-4029-bb2d-293d6f337343 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:51.192202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:51.205341 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 390/779] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:21:51.228436 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40916c4c-70b6-43bc-98c9-18e97fd3f749 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:51.290902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90d05f-0a27-4029-bb2d-293d6f337343 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:51.383631 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4498b76-7c69-477b-b040-c3acf059dd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:51.384239 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4498b76-7c69-477b-b040-c3acf059dd5b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:51.384535 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4498b76-7c69-477b-b040-c3acf059dd5b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:51.384825 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4498b76-7c69-477b-b040-c3acf059dd5b None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:51.385588 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4498b76-7c69-477b-b040-c3acf059dd5b None admin] RBAC: Adding query filter params (type=network) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:51.385906 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4498b76-7c69-477b-b040-c3acf059dd5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:51.387378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4498b76-7c69-477b-b040-c3acf059dd5b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:51.389994 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40916c4c-70b6-43bc-98c9-18e97fd3f749 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:51.408937 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 390/780] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 17:21:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:51.414776 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4498b76-7c69-477b-b040-c3acf059dd5b None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:51.456179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77cfd7-34ef-440c-8a32-353e66a58097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:51.457002 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77cfd7-34ef-440c-8a32-353e66a58097 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:51.457329 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77cfd7-34ef-440c-8a32-353e66a58097 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:51.457613 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77cfd7-34ef-440c-8a32-353e66a58097 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:51.473615 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:51.491149 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad77cfd7-34ef-440c-8a32-353e66a58097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:51.498942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-ad77cfd7-34ef-440c-8a32-353e66a58097 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:21:51.498942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:51.498942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:51.498942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:51.498942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:51.498942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:51.498942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:51.498942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:51.498942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:51.498942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:51.498942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:51.498942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:51.498942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:51.498942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:51.498942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:51.498942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:51.500124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:51.500124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:51.500124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:51.500124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:51.500124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:51.500124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:51.500124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:51.500124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:51.500124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:51.500124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:51.500124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:51.500124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:51.500124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:51.500124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:51.500124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:51.500124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:51.501080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:51.501080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:51.501080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:51.501080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:51.501080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:51.501080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:51.501080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:51.501080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:51.501080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:51.501080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:51.501080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:21:51.501080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:51.509596 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 391/781] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:21:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:21:51.516057 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77cfd7-34ef-440c-8a32-353e66a58097 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:51.533486 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4498b76-7c69-477b-b040-c3acf059dd5b None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:51.603406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f70310-d380-4413-aa25-208195ad63f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:51.604161 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f70310-d380-4413-aa25-208195ad63f5 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:51.604494 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f70310-d380-4413-aa25-208195ad63f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:51.604830 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f70310-d380-4413-aa25-208195ad63f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:51.605818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f70310-d380-4413-aa25-208195ad63f5 None admin] RBAC: Adding query filter params (interface=public, service_id=33d4a4f050dc44a58aae73865093b42d, region_id=RegionOne) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:51.606209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f70310-d380-4413-aa25-208195ad63f5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:51.607900 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f70310-d380-4413-aa25-208195ad63f5 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:51.621445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:51.626897 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 391/782] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jun 16 17:21:51 2022] GET /identity/v3/endpoints?service_id=33d4a4f050dc44a58aae73865093b42d&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:51.632220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f70310-d380-4413-aa25-208195ad63f5 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:51.691263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77cfd7-34ef-440c-8a32-353e66a58097 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:51.749076 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d29c8e-8d52-4676-9080-10dde75f8b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:51.751384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d29c8e-8d52-4676-9080-10dde75f8b16 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:51.753361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d29c8e-8d52-4676-9080-10dde75f8b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:51.753678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d29c8e-8d52-4676-9080-10dde75f8b16 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:51.754559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d29c8e-8d52-4676-9080-10dde75f8b16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:51.754888 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d29c8e-8d52-4676-9080-10dde75f8b16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:51.756385 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d29c8e-8d52-4676-9080-10dde75f8b16 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:51.783633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 392/783] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:51.794912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d29c8e-8d52-4676-9080-10dde75f8b16 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:51.825744 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f70310-d380-4413-aa25-208195ad63f5 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:51.841290 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:51.864695 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc38c8-dc97-4b5a-b01a-1cdccb10377f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:51.865521 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc38c8-dc97-4b5a-b01a-1cdccb10377f None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:51.865818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc38c8-dc97-4b5a-b01a-1cdccb10377f None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:51.866108 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc38c8-dc97-4b5a-b01a-1cdccb10377f None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:51.866875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc38c8-dc97-4b5a-b01a-1cdccb10377f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:51.868318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc38c8-dc97-4b5a-b01a-1cdccb10377f None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:51.874281 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 392/784] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jun 16 17:21:51 2022] GET /identity/v3/services => generated 772 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:51.876361 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600638b4-30ea-436a-87d3-abf8d186ab7f None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:51.876729 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600638b4-30ea-436a-87d3-abf8d186ab7f None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:51.877081 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600638b4-30ea-436a-87d3-abf8d186ab7f None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:51.879025 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 393/785] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:51.892655 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31af33-c69e-4d90-8bc4-c9120ca7e08c None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:51.892980 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31af33-c69e-4d90-8bc4-c9120ca7e08c None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:51.893258 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31af33-c69e-4d90-8bc4-c9120ca7e08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:51.926342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-9f31af33-c69e-4d90-8bc4-c9120ca7e08c None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:51.927562 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-9f31af33-c69e-4d90-8bc4-c9120ca7e08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:51.952520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d29c8e-8d52-4676-9080-10dde75f8b16 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:51.970452 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:52.004903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 394/786] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:21:52.005634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9fc685-7fbd-4d82-8626-d53d16f1bb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:52.006712 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9fc685-7fbd-4d82-8626-d53d16f1bb05 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:52.007260 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9fc685-7fbd-4d82-8626-d53d16f1bb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:52.007751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9fc685-7fbd-4d82-8626-d53d16f1bb05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:52.033853 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9fc685-7fbd-4d82-8626-d53d16f1bb05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:52.035743 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9fc685-7fbd-4d82-8626-d53d16f1bb05 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:52.061099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-9f9fc685-7fbd-4d82-8626-d53d16f1bb05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:52.061099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:52.061099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:52.061099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:52.061099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:52.061099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:52.061099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:52.061099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:52.061099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:52.061099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:52.061099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:52.061099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:52.061099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:52.061099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:52.061099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:52.062877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:52.062877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:52.062877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:52.062877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:52.062877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:52.062877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:52.062877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:52.062877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:52.062877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:52.062877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:52.062877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:52.062877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:52.062877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:52.062877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:52.062877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:52.062877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:52.062877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:52.064046 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:52.064046 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:52.064046 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:52.064046 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:52.066524 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 393/787] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:21:52.068620 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31af33-c69e-4d90-8bc4-c9120ca7e08c None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:52.075688 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9fc685-7fbd-4d82-8626-d53d16f1bb05 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:52.205247 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31af33-c69e-4d90-8bc4-c9120ca7e08c None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:52.214981 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:52.229172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9fc685-7fbd-4d82-8626-d53d16f1bb05 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:52.257313 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd9aae-6dbd-4c82-9983-20d06e33d09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:52.258726 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd9aae-6dbd-4c82-9983-20d06e33d09b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:52.259200 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd9aae-6dbd-4c82-9983-20d06e33d09b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:52.259727 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd9aae-6dbd-4c82-9983-20d06e33d09b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:52.266136 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3442b-addc-49b9-9c05-dd5a905b3c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:52.266810 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3442b-addc-49b9-9c05-dd5a905b3c9a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:52.267090 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3442b-addc-49b9-9c05-dd5a905b3c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:52.267386 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3442b-addc-49b9-9c05-dd5a905b3c9a None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:52.268101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3442b-addc-49b9-9c05-dd5a905b3c9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:52.268437 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3442b-addc-49b9-9c05-dd5a905b3c9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:52.269939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3442b-addc-49b9-9c05-dd5a905b3c9a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:52.274078 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd9aae-6dbd-4c82-9983-20d06e33d09b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:52.283220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-6ffd9aae-6dbd-4c82-9983-20d06e33d09b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:21:52.283220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:52.283220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:52.283220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:52.283220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:52.283220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:52.283220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:52.283220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:52.283220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:52.283220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:52.283220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:52.283220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:52.283220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:52.283220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:52.283220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:52.283220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:52.284363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:52.284363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:52.284363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:52.284363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:52.284363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:52.284363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:52.284363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:52.284363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:52.284363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:52.284363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:52.284363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:52.284363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:52.284363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:52.284363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:52.284363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:52.284363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:52.285317 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:52.285317 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:52.285317 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:52.285317 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:52.285317 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:52.285317 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:52.285317 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:52.285317 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:52.285317 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:52.285317 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:52.285317 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:21:52.285317 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:52.285979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 394/788] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:52.287603 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 395/789] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:21:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:52.290816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3442b-addc-49b9-9c05-dd5a905b3c9a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:52.293077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd9aae-6dbd-4c82-9983-20d06e33d09b None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:52.320450 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:52.400935 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3442b-addc-49b9-9c05-dd5a905b3c9a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:52.406396 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd9aae-6dbd-4c82-9983-20d06e33d09b None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:52.459719 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a78fe-1e61-4897-a133-fb5d47503bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:52.460778 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a78fe-1e61-4897-a133-fb5d47503bfc None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:52.461271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a78fe-1e61-4897-a133-fb5d47503bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:52.461698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a78fe-1e61-4897-a133-fb5d47503bfc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:52.470489 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a78fe-1e61-4897-a133-fb5d47503bfc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:52.472146 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a78fe-1e61-4897-a133-fb5d47503bfc None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:52.486515 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7884b77-f082-4b5b-8c43-91c3eae9f813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:52.487209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7884b77-f082-4b5b-8c43-91c3eae9f813 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:52.487501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7884b77-f082-4b5b-8c43-91c3eae9f813 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:52.487790 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7884b77-f082-4b5b-8c43-91c3eae9f813 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:52.488376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-d38a78fe-1e61-4897-a133-fb5d47503bfc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:21:52.488376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:52.488376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:52.488376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:52.488376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:52.488376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:52.488376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:52.488376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:52.488376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:52.488376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:52.488376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:52.488376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:52.488376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:52.488376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:52.488376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:52.488376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:52.489629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:52.489629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:52.489629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:52.489629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:52.489629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:52.489629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:52.489629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:52.489629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:52.489629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:52.489629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:52.489629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:52.489629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:52.489629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:52.489629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:52.489629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:52.489629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:52.489629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:52.489629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:52.490757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:52.490757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:52.490757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:52.490757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:52.490757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:52.490757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:52.490757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:52.490757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:52.490757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:21:52.490757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:52.491401 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7884b77-f082-4b5b-8c43-91c3eae9f813 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:52.491401 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7884b77-f082-4b5b-8c43-91c3eae9f813 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:52.491401 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7884b77-f082-4b5b-8c43-91c3eae9f813 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:52.493709 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 395/790] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:21:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:52.498951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a78fe-1e61-4897-a133-fb5d47503bfc None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:52.514926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 396/791] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:52.516698 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7884b77-f082-4b5b-8c43-91c3eae9f813 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:52.556610 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:52.631146 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7884b77-f082-4b5b-8c43-91c3eae9f813 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:52.677202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:52.687568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ead13-2185-40f9-b0af-ad4bcecf1a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:52.688354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ead13-2185-40f9-b0af-ad4bcecf1a5f None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:52.688648 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ead13-2185-40f9-b0af-ad4bcecf1a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:52.688929 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ead13-2185-40f9-b0af-ad4bcecf1a5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:52.702983 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a78fe-1e61-4897-a133-fb5d47503bfc None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:52.713761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ead13-2185-40f9-b0af-ad4bcecf1a5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:52.715531 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ead13-2185-40f9-b0af-ad4bcecf1a5f None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:52.745994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-5e9ead13-2185-40f9-b0af-ad4bcecf1a5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:52.745994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:52.745994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:52.745994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:52.745994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:52.745994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:52.745994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:52.745994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:52.745994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:52.745994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:52.745994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:52.745994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:52.745994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:52.745994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:52.745994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:52.746969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:52.746969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:52.746969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:52.746969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:52.746969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:52.746969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:52.746969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:52.746969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:52.746969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:52.746969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:52.746969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:52.746969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:52.746969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:52.746969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:52.746969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:52.746969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:52.746969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:52.747868 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:52.747868 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:52.747868 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:52.747868 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:52.748524 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 397/792] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:52.755106 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ead13-2185-40f9-b0af-ad4bcecf1a5f None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:52.762453 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3251e-05ca-45c1-9c6c-ee77bea475d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:52.763142 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3251e-05ca-45c1-9c6c-ee77bea475d9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:52.763431 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3251e-05ca-45c1-9c6c-ee77bea475d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:52.763711 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3251e-05ca-45c1-9c6c-ee77bea475d9 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:52.764560 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc3251e-05ca-45c1-9c6c-ee77bea475d9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:52.764877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc3251e-05ca-45c1-9c6c-ee77bea475d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:52.767266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc3251e-05ca-45c1-9c6c-ee77bea475d9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:52.803580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 396/793] 10.222.0.40 () {56 vars in 1135 bytes} [Thu Jun 16 17:21:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:52.806899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3251e-05ca-45c1-9c6c-ee77bea475d9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:52.895882 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:52.932171 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ead13-2185-40f9-b0af-ad4bcecf1a5f None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:52.997327 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ffbec-a466-447d-9012-05438effc823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:52.998458 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ffbec-a466-447d-9012-05438effc823 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:52.998752 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ffbec-a466-447d-9012-05438effc823 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:52.999039 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ffbec-a466-447d-9012-05438effc823 None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:52.999875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ffbec-a466-447d-9012-05438effc823 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:53.000194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ffbec-a466-447d-9012-05438effc823 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:53.001620 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ffbec-a466-447d-9012-05438effc823 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:53.017357 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 398/794] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:53.027094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3251e-05ca-45c1-9c6c-ee77bea475d9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:53.030271 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ffbec-a466-447d-9012-05438effc823 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:53.071803 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb611a5d-27c9-4f5a-8edd-d86798507351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:53.072624 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb611a5d-27c9-4f5a-8edd-d86798507351 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:53.072916 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb611a5d-27c9-4f5a-8edd-d86798507351 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:53.073242 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb611a5d-27c9-4f5a-8edd-d86798507351 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:53.096845 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb611a5d-27c9-4f5a-8edd-d86798507351 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:53.098345 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb611a5d-27c9-4f5a-8edd-d86798507351 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:53.106021 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:53.129965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-cb611a5d-27c9-4f5a-8edd-d86798507351 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:53.129965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:53.129965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:53.129965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:53.129965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:53.129965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:53.129965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:53.129965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:53.129965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:53.129965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:53.129965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:53.129965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:53.129965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:53.129965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:53.129965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:53.130928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:53.130928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:53.130928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:53.130928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:53.130928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:53.130928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:53.130928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:53.130928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:53.130928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:53.130928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:53.130928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:53.130928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:53.130928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:53.130928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:53.130928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:53.130928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:53.130928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:53.131862 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:53.131862 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:53.131862 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:53.131862 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:53.131862 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 397/795] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:53.141276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb611a5d-27c9-4f5a-8edd-d86798507351 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:53.196589 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ffbec-a466-447d-9012-05438effc823 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:53.260279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e8f7e-3c3e-42dc-88e9-aeee2cafc4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:53.261406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e8f7e-3c3e-42dc-88e9-aeee2cafc4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:53.261847 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e8f7e-3c3e-42dc-88e9-aeee2cafc4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:53.262300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e8f7e-3c3e-42dc-88e9-aeee2cafc4c0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:53.272627 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17e8f7e-3c3e-42dc-88e9-aeee2cafc4c0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:53.274574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17e8f7e-3c3e-42dc-88e9-aeee2cafc4c0 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:53.277810 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:53.293743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-e17e8f7e-3c3e-42dc-88e9-aeee2cafc4c0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:21:53.293743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:53.293743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:53.293743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:53.293743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:53.293743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:53.293743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:53.293743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:53.293743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:53.293743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:53.293743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:53.293743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:53.293743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:53.293743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:53.293743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:53.293743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:53.294982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:53.294982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:53.294982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:53.294982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:53.294982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:53.294982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:53.294982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:53.294982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:53.294982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:53.294982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:53.294982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:53.294982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:53.294982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:53.294982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:53.294982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:53.294982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:53.294982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:53.294982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:53.296175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:53.296175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:53.296175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:53.296175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:53.296175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:53.296175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:53.296175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:53.296175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:53.296175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:21:53.296175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:53.299108 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 399/796] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jun 16 17:21:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:53.306555 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e8f7e-3c3e-42dc-88e9-aeee2cafc4c0 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:53.354018 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb611a5d-27c9-4f5a-8edd-d86798507351 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:53.395483 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d183bcf-5e17-41dd-8cd4-7f42a2752e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:53.396440 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d183bcf-5e17-41dd-8cd4-7f42a2752e62 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:53.396877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d183bcf-5e17-41dd-8cd4-7f42a2752e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:53.397394 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d183bcf-5e17-41dd-8cd4-7f42a2752e62 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:53.398368 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d183bcf-5e17-41dd-8cd4-7f42a2752e62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:53.398827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d183bcf-5e17-41dd-8cd4-7f42a2752e62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:53.400554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d183bcf-5e17-41dd-8cd4-7f42a2752e62 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:53.414694 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 398/797] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:53.416339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28254113-9c5b-474b-9a37-93af14d9e2b3 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:53.416843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28254113-9c5b-474b-9a37-93af14d9e2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:53.417378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28254113-9c5b-474b-9a37-93af14d9e2b3 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:53.419150 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 399/798] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:53.420223 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28254113-9c5b-474b-9a37-93af14d9e2b3 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:53.423500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e8f7e-3c3e-42dc-88e9-aeee2cafc4c0 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:53.464783 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9db0ba-567e-45cf-aa6b-2733d51e8ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:53.465527 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9db0ba-567e-45cf-aa6b-2733d51e8ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:53.465820 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9db0ba-567e-45cf-aa6b-2733d51e8ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:53.466106 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9db0ba-567e-45cf-aa6b-2733d51e8ea1 None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:53.466996 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9db0ba-567e-45cf-aa6b-2733d51e8ea1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:53.467316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9db0ba-567e-45cf-aa6b-2733d51e8ea1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:53.469745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9db0ba-567e-45cf-aa6b-2733d51e8ea1 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:53.491040 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 400/799] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jun 16 17:21:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:53.492178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:53.492493 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44b7e6-db4e-4a82-900e-a5179300c9a6 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:53.492847 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44b7e6-db4e-4a82-900e-a5179300c9a6 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:53.493505 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44b7e6-db4e-4a82-900e-a5179300c9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:53.511623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-cc44b7e6-db4e-4a82-900e-a5179300c9a6 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:53.512501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-cc44b7e6-db4e-4a82-900e-a5179300c9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:53.548668 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28254113-9c5b-474b-9a37-93af14d9e2b3 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:53.569876 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:53.585116 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d3194-fb9f-4f3e-909e-6af9e7eec2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:53.586048 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d3194-fb9f-4f3e-909e-6af9e7eec2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:53.586344 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d3194-fb9f-4f3e-909e-6af9e7eec2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:53.586627 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d3194-fb9f-4f3e-909e-6af9e7eec2a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:53.597461 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 401/800] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:21:53.597461 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44b7e6-db4e-4a82-900e-a5179300c9a6 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:53.597674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d3194-fb9f-4f3e-909e-6af9e7eec2a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:53.597674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d3194-fb9f-4f3e-909e-6af9e7eec2a3 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:53.610212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-3f9d3194-fb9f-4f3e-909e-6af9e7eec2a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:53.610212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:53.610212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:53.610212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:53.610212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:53.610212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:53.610212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:53.610212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:53.610212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:53.610212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:53.610212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:53.610212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:53.610212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:53.610212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:53.610212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:53.611170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:53.611170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:53.611170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:53.611170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:53.611170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:53.611170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:53.611170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:53.611170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:53.611170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:53.611170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:53.611170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:53.611170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:53.611170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:53.611170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:53.611170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:53.611170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:53.611170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:53.612120 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:53.612120 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:53.612120 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:53.612120 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:53.612120 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:53.612120 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:53.612120 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:53.612120 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:53.612120 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 400/801] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:53.613833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847db85f-f6fd-44a0-95ea-dd8d8f806703 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:53.614335 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847db85f-f6fd-44a0-95ea-dd8d8f806703 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:53.614735 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847db85f-f6fd-44a0-95ea-dd8d8f806703 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:53.619907 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 401/802] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:53.619907 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847db85f-f6fd-44a0-95ea-dd8d8f806703 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:53.707073 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44b7e6-db4e-4a82-900e-a5179300c9a6 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:53.727049 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847db85f-f6fd-44a0-95ea-dd8d8f806703 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:53.741042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4d131-dbe0-4466-9e0e-7597936872f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:53.741719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4d131-dbe0-4466-9e0e-7597936872f1 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:53.742008 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4d131-dbe0-4466-9e0e-7597936872f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:53.742299 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4d131-dbe0-4466-9e0e-7597936872f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:53.744068 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:53.763951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758bd5e2-f9bb-4c80-9f53-ceb6f1d9a68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:53.764548 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758bd5e2-f9bb-4c80-9f53-ceb6f1d9a68d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:53.764841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758bd5e2-f9bb-4c80-9f53-ceb6f1d9a68d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:53.765783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758bd5e2-f9bb-4c80-9f53-ceb6f1d9a68d None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:53.766582 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758bd5e2-f9bb-4c80-9f53-ceb6f1d9a68d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:53.766928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758bd5e2-f9bb-4c80-9f53-ceb6f1d9a68d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:53.768359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758bd5e2-f9bb-4c80-9f53-ceb6f1d9a68d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:53.780286 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 402/803] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:53.781253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758bd5e2-f9bb-4c80-9f53-ceb6f1d9a68d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:53.806710 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:53.820972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4d131-dbe0-4466-9e0e-7597936872f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:53.822461 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4d131-dbe0-4466-9e0e-7597936872f1 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:53.885287 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758bd5e2-f9bb-4c80-9f53-ceb6f1d9a68d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:53.920961 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3aa857-9829-499f-b45a-5966e3e69796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:53.921831 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3aa857-9829-499f-b45a-5966e3e69796 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:53.922368 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3aa857-9829-499f-b45a-5966e3e69796 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:53.922368 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3aa857-9829-499f-b45a-5966e3e69796 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:53.933259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 402/804] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:21:53.933259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad12537-96fc-48f5-8d8c-9a1a0fc67574 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:53.933259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad12537-96fc-48f5-8d8c-9a1a0fc67574 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:53.933259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad12537-96fc-48f5-8d8c-9a1a0fc67574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:53.938104 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3aa857-9829-499f-b45a-5966e3e69796 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:53.945900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-fa3aa857-9829-499f-b45a-5966e3e69796 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:53.945900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:53.945900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:53.945900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:53.945900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:53.945900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:53.945900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:53.945900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:53.945900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:53.945900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:53.945900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:53.945900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:53.945900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:53.945900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:53.945900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:53.945900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:53.946844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:53.946844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:53.946844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:53.946844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:53.946844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:53.946844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:53.946844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:53.946844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:53.946844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:53.946844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:53.946844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:53.946844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:53.946844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:53.946844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:53.946844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:53.946844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:53.953532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:53.953532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:53.953532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:53.953532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:53.953532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:53.953532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:53.953532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:53.953532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:53.953532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:53.953532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:53.953532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:53.953532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:53.953532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 403/805] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:21:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:53.954257 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-2ad12537-96fc-48f5-8d8c-9a1a0fc67574 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:53.954338 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3aa857-9829-499f-b45a-5966e3e69796 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:53.954410 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-2ad12537-96fc-48f5-8d8c-9a1a0fc67574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:53.991006 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:54.041238 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 403/806] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:21:54.041238 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad12537-96fc-48f5-8d8c-9a1a0fc67574 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:54.061010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:54.071835 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3aa857-9829-499f-b45a-5966e3e69796 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:54.108798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49ba6c-9d73-4459-afed-c40a2371a82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:54.109773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49ba6c-9d73-4459-afed-c40a2371a82c None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:54.110053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49ba6c-9d73-4459-afed-c40a2371a82c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:54.110335 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49ba6c-9d73-4459-afed-c40a2371a82c None admin] PATH_INFO: `/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/b8a37d1201d54afdad9b228ef4f2d787/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:54.112583 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49ba6c-9d73-4459-afed-c40a2371a82c None admin] RBAC: Authorizing `identity:create_grant(project_id=35a535a305e740368b7a4d77d0ad22a0, user_id=b8a37d1201d54afdad9b228ef4f2d787, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:54.114267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49ba6c-9d73-4459-afed-c40a2371a82c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:54.137194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 404/807] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:21:53 2022] PUT /identity/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/b8a37d1201d54afdad9b228ef4f2d787/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:21:54.137194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49ba6c-9d73-4459-afed-c40a2371a82c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:54.161331 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad12537-96fc-48f5-8d8c-9a1a0fc67574 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:54.206172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b3e8b-5452-478a-99d9-7d3a12496b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:54.207090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b3e8b-5452-478a-99d9-7d3a12496b02 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:54.207446 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b3e8b-5452-478a-99d9-7d3a12496b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:54.207903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b3e8b-5452-478a-99d9-7d3a12496b02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:54.220589 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b3e8b-5452-478a-99d9-7d3a12496b02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:54.222469 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b3e8b-5452-478a-99d9-7d3a12496b02 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:54.251049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-361b3e8b-5452-478a-99d9-7d3a12496b02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:54.251049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:54.251049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:54.251049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:54.251049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:54.251049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:54.251049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:54.251049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:54.251049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:54.251049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:54.251049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:54.251049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:54.251049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:54.251049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:54.251049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:54.252018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:54.252018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:54.252018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:54.252018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:54.252018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:54.252018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:54.252018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:54.252018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:54.252018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:54.252018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:54.252018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:54.252018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:54.252018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:54.252018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:54.252018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:54.252018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:54.252018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:54.252951 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:54.252951 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:54.252951 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:54.252951 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:54.252951 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 404/808] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:54.257857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b3e8b-5452-478a-99d9-7d3a12496b02 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:54.271141 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:54.279552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49ba6c-9d73-4459-afed-c40a2371a82c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:54.326347 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c79ba-dd8f-4528-8db2-72b72ac0ce47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:54.327251 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c79ba-dd8f-4528-8db2-72b72ac0ce47 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:54.328316 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c79ba-dd8f-4528-8db2-72b72ac0ce47 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:54.328757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c79ba-dd8f-4528-8db2-72b72ac0ce47 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:54.329702 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c79ba-dd8f-4528-8db2-72b72ac0ce47 None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:54.330163 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c79ba-dd8f-4528-8db2-72b72ac0ce47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:54.331656 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c79ba-dd8f-4528-8db2-72b72ac0ce47 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:54.343554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 405/809] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 17:21:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:54.344432 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c79ba-dd8f-4528-8db2-72b72ac0ce47 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:54.357864 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:54.396584 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b3e8b-5452-478a-99d9-7d3a12496b02 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:54.435357 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3a7de-99fc-4e4e-baec-3b672936bb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:54.436488 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3a7de-99fc-4e4e-baec-3b672936bb63 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:54.436853 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3a7de-99fc-4e4e-baec-3b672936bb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:54.437141 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3a7de-99fc-4e4e-baec-3b672936bb63 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:54.437891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3a7de-99fc-4e4e-baec-3b672936bb63 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:54.439465 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3a7de-99fc-4e4e-baec-3b672936bb63 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:54.457346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-5ea3a7de-99fc-4e4e-baec-3b672936bb63 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:21:54.457346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:54.457346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:21:54.457346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:21:54.457346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:21:54.457346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:21:54.457346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:21:54.457346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:21:54.457346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:21:54.457346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:54.457346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:21:54.457346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:54.457346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:54.457346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:54.457346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:54.457346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:54.458384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:54.458384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:54.458384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:54.458384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:54.458384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:54.458384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:54.458384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:54.458384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:21:54.458384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:21:54.458384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:21:54.458384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:21:54.458384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:54.458384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:54.458384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:54.458384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:54.458384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:54.459329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:54.459329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:54.459329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:54.459329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:54.459329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:54.459329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:54.459329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:54.459329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:54.459329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:54.459329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:54.459329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:54.459329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:21:54.459329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:21:54.459329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:21:54.459329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:54.459329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 405/810] 10.222.0.40 () {56 vars in 1129 bytes} [Thu Jun 16 17:21:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:54.460208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3a7de-99fc-4e4e-baec-3b672936bb63 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:54.479407 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c79ba-dd8f-4528-8db2-72b72ac0ce47 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:54.528265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a112bf8-7f45-4f8a-a533-91e797880b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:54.528943 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a112bf8-7f45-4f8a-a533-91e797880b58 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:54.529262 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a112bf8-7f45-4f8a-a533-91e797880b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:54.529544 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a112bf8-7f45-4f8a-a533-91e797880b58 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:54.530763 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a112bf8-7f45-4f8a-a533-91e797880b58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:54.531187 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a112bf8-7f45-4f8a-a533-91e797880b58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:54.532849 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a112bf8-7f45-4f8a-a533-91e797880b58 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:54.551803 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 406/811] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:54.551803 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a112bf8-7f45-4f8a-a533-91e797880b58 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:54.572788 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:54.582292 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3a7de-99fc-4e4e-baec-3b672936bb63 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:54.618439 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeaeafc-cefa-4c7a-b568-fd4b691a5062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:54.619266 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeaeafc-cefa-4c7a-b568-fd4b691a5062 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:54.619551 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeaeafc-cefa-4c7a-b568-fd4b691a5062 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:54.619828 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeaeafc-cefa-4c7a-b568-fd4b691a5062 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:54.622139 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:54.632265 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeaeafc-cefa-4c7a-b568-fd4b691a5062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:54.634001 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeaeafc-cefa-4c7a-b568-fd4b691a5062 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:54.652120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-bfeaeafc-cefa-4c7a-b568-fd4b691a5062 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:54.652120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:54.652120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:54.652120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:54.652120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:54.652120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:54.652120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:54.652120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:54.652120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:54.652120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:54.652120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:54.652120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:54.652120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:54.652120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:54.652120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:54.653085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:54.653085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:54.653085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:54.653085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:54.653085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:54.653085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:54.653085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:54.653085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:54.653085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:54.653085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:54.653085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:54.653085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:54.653085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:54.653085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:54.653085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:54.653085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:54.653085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:54.654051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:54.654051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:54.654051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:54.654051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:54.658058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 406/812] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:54.659325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38282993-e858-4101-b7b3-e4bcfb61a282 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:54.659625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38282993-e858-4101-b7b3-e4bcfb61a282 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:54.659913 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38282993-e858-4101-b7b3-e4bcfb61a282 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:54.661522 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 407/813] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:54.662221 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38282993-e858-4101-b7b3-e4bcfb61a282 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:54.667228 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a112bf8-7f45-4f8a-a533-91e797880b58 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:54.704708 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2955f7-402f-4f32-b7e2-9500d26f67f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:54.705414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2955f7-402f-4f32-b7e2-9500d26f67f1 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:54.705704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2955f7-402f-4f32-b7e2-9500d26f67f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:54.705983 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2955f7-402f-4f32-b7e2-9500d26f67f1 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:54.706695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2955f7-402f-4f32-b7e2-9500d26f67f1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:54.707005 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2955f7-402f-4f32-b7e2-9500d26f67f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:54.708377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2955f7-402f-4f32-b7e2-9500d26f67f1 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:54.717410 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 407/814] 10.222.0.40 () {56 vars in 1124 bytes} [Thu Jun 16 17:21:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:54.717410 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2955f7-402f-4f32-b7e2-9500d26f67f1 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:54.789425 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38282993-e858-4101-b7b3-e4bcfb61a282 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:54.803208 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:54.823898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2955f7-402f-4f32-b7e2-9500d26f67f1 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:54.825408 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9580e3-76d1-43e9-9d27-89d6f9526cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:54.826211 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9580e3-76d1-43e9-9d27-89d6f9526cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:54.826498 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9580e3-76d1-43e9-9d27-89d6f9526cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:54.826778 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9580e3-76d1-43e9-9d27-89d6f9526cc9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:54.834766 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9580e3-76d1-43e9-9d27-89d6f9526cc9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:54.836138 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9580e3-76d1-43e9-9d27-89d6f9526cc9 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:54.851621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-4d9580e3-76d1-43e9-9d27-89d6f9526cc9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:54.851621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:54.851621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:54.851621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:54.851621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:54.851621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:54.851621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:54.851621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:54.851621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:54.851621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:54.851621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:54.851621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:54.851621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:54.851621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:54.851621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:54.852574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:54.852574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:54.852574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:54.852574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:54.852574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:54.852574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:54.852574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:54.852574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:54.852574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:54.852574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:54.852574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:54.852574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:54.852574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:54.852574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:54.852574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:54.852574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:54.852574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:54.853941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:54.853941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:54.853941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:54.853941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:54.853941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:54.853941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:54.853941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:54.853941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:54.853941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 408/815] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:54.854448 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb8744-65eb-421b-a7e8-b2d4d835a5af None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:54.854448 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb8744-65eb-421b-a7e8-b2d4d835a5af None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:54.854918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb8744-65eb-421b-a7e8-b2d4d835a5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:54.870984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045bdd2-c578-4bce-84ca-2320ad2e7035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:54.872082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045bdd2-c578-4bce-84ca-2320ad2e7035 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:54.872198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-0cdb8744-65eb-421b-a7e8-b2d4d835a5af None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:54.872291 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045bdd2-c578-4bce-84ca-2320ad2e7035 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:54.872291 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045bdd2-c578-4bce-84ca-2320ad2e7035 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:54.872868 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0cdb8744-65eb-421b-a7e8-b2d4d835a5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:54.872981 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045bdd2-c578-4bce-84ca-2320ad2e7035 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:54.873277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045bdd2-c578-4bce-84ca-2320ad2e7035 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:54.875743 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045bdd2-c578-4bce-84ca-2320ad2e7035 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:54.888043 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 408/816] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:54.888768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045bdd2-c578-4bce-84ca-2320ad2e7035 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:54.925450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:54.953006 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 409/817] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:21:54.954188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb8744-65eb-421b-a7e8-b2d4d835a5af None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:55.019337 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045bdd2-c578-4bce-84ca-2320ad2e7035 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:55.055589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360568d-fef5-4ed5-bb18-ba5fee7d9fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:55.056519 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360568d-fef5-4ed5-bb18-ba5fee7d9fec None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:55.056957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360568d-fef5-4ed5-bb18-ba5fee7d9fec None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:55.057464 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360568d-fef5-4ed5-bb18-ba5fee7d9fec None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:55.058438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360568d-fef5-4ed5-bb18-ba5fee7d9fec None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:55.058896 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360568d-fef5-4ed5-bb18-ba5fee7d9fec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:55.060438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360568d-fef5-4ed5-bb18-ba5fee7d9fec None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:55.060623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb8744-65eb-421b-a7e8-b2d4d835a5af None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:55.066331 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 409/818] 10.222.0.40 () {56 vars in 1124 bytes} [Thu Jun 16 17:21:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:55.067270 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360568d-fef5-4ed5-bb18-ba5fee7d9fec None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:55.080338 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:55.100235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda041d1-d469-4d71-8697-3091fbf7e405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:55.100838 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda041d1-d469-4d71-8697-3091fbf7e405 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:55.101129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda041d1-d469-4d71-8697-3091fbf7e405 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:55.101442 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda041d1-d469-4d71-8697-3091fbf7e405 None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:55.102252 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda041d1-d469-4d71-8697-3091fbf7e405 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:55.102553 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda041d1-d469-4d71-8697-3091fbf7e405 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:55.103982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda041d1-d469-4d71-8697-3091fbf7e405 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:55.122245 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 410/819] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:55.123046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda041d1-d469-4d71-8697-3091fbf7e405 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:55.163224 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:55.180181 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360568d-fef5-4ed5-bb18-ba5fee7d9fec None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:55.215579 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0427683-4ef5-44a5-9601-b9accb3b348f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:55.216747 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0427683-4ef5-44a5-9601-b9accb3b348f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:55.217263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0427683-4ef5-44a5-9601-b9accb3b348f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:55.217772 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0427683-4ef5-44a5-9601-b9accb3b348f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:55.226945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0427683-4ef5-44a5-9601-b9accb3b348f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:55.228520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0427683-4ef5-44a5-9601-b9accb3b348f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:55.242420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda041d1-d469-4d71-8697-3091fbf7e405 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:55.251827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-c0427683-4ef5-44a5-9601-b9accb3b348f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:21:55.251827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:55.251827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:55.251827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:55.251827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:55.251827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:55.251827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:55.251827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:55.251827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:55.251827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:55.251827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:55.251827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:55.251827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:55.251827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:55.251827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:55.251827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:55.252818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:55.252818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:55.252818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:55.252818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:55.252818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:55.252818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:55.252818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:55.252818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:55.252818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:55.252818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:55.252818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:55.252818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:55.252818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:55.252818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:55.252818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:55.252818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:55.252818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:55.252818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:55.253925 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:55.253925 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:55.253925 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:55.253925 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:55.253925 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:55.253925 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:55.253925 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:55.253925 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:55.253925 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:21:55.253925 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:55.255153 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 410/820] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jun 16 17:21:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:55.255995 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0427683-4ef5-44a5-9601-b9accb3b348f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:55.283864 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bbc10a-6781-448c-89d8-f3c00ac1c18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:55.285070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbc10a-6781-448c-89d8-f3c00ac1c18c None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:55.285589 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbc10a-6781-448c-89d8-f3c00ac1c18c None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:55.286067 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbc10a-6781-448c-89d8-f3c00ac1c18c None admin] PATH_INFO: `/v3/services/network` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:55.286743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bbc10a-6781-448c-89d8-f3c00ac1c18c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:55.288274 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bbc10a-6781-448c-89d8-f3c00ac1c18c None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:55.305235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-68bbc10a-6781-448c-89d8-f3c00ac1c18c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:21:55.305235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:55.305235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:21:55.305235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:21:55.305235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:21:55.305235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:21:55.305235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:21:55.305235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:21:55.305235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:21:55.305235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:55.305235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:21:55.305235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:55.305235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:55.305235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:55.305235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:55.305235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:55.305235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:55.306350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:55.306350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:55.306350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:55.306350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:55.306350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:55.306350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:55.306350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:21:55.306350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:21:55.306350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:21:55.306350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:21:55.306350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:55.306350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:55.306350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:55.306350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:55.306350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:55.306350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:55.307237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:55.307237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:55.307237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:55.307237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:55.307237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:55.307237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:55.307237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:55.307237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:55.307237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:55.307237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:55.307237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:21:55.307237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:21:55.307237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:21:55.307237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:55.307237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 411/821] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:55 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:55.308780 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bbc10a-6781-448c-89d8-f3c00ac1c18c None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:55.332729 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:55.370707 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0427683-4ef5-44a5-9601-b9accb3b348f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:55.410263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94052bd-dd91-4108-ab34-72b37ab76b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:55.410970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94052bd-dd91-4108-ab34-72b37ab76b04 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:55.411271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94052bd-dd91-4108-ab34-72b37ab76b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:55.411556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94052bd-dd91-4108-ab34-72b37ab76b04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:55.412321 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94052bd-dd91-4108-ab34-72b37ab76b04 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:55.413821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94052bd-dd91-4108-ab34-72b37ab76b04 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:55.416296 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:55.419040 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bbc10a-6781-448c-89d8-f3c00ac1c18c None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:55.445186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 411/822] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jun 16 17:21:55 2022] POST /identity/v3/endpoints => generated 346 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:21:55.445186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94052bd-dd91-4108-ab34-72b37ab76b04 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:55.475633 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b032702-12a3-473a-906a-182e05762bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:55.476301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b032702-12a3-473a-906a-182e05762bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:55.476587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b032702-12a3-473a-906a-182e05762bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:55.476864 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b032702-12a3-473a-906a-182e05762bf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:55.477762 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b032702-12a3-473a-906a-182e05762bf4 None admin] RBAC: Adding query filter params (role.id=25a16d8ea4084e9eacdebb7a4dcd2337, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=689a31e26c6b454a91c3fd79678b4d07) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:55.478079 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b032702-12a3-473a-906a-182e05762bf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:55.479791 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b032702-12a3-473a-906a-182e05762bf4 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:55.493376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 412/823] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:21:55 2022] GET /identity/v3/role_assignments?user.id=689a31e26c6b454a91c3fd79678b4d07&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=25a16d8ea4084e9eacdebb7a4dcd2337 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:55.494410 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b032702-12a3-473a-906a-182e05762bf4 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:55.640568 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94052bd-dd91-4108-ab34-72b37ab76b04 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:55.657644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b032702-12a3-473a-906a-182e05762bf4 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:55.683453 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:55.700627 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bf131-69bb-4f84-b5fb-d5c329ece5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:55.701494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bf131-69bb-4f84-b5fb-d5c329ece5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:55.701884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bf131-69bb-4f84-b5fb-d5c329ece5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:55.702217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bf131-69bb-4f84-b5fb-d5c329ece5d4 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:55.704682 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bf131-69bb-4f84-b5fb-d5c329ece5d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:55.705081 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bf131-69bb-4f84-b5fb-d5c329ece5d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:55.706792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bf131-69bb-4f84-b5fb-d5c329ece5d4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:55.712035 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c426b-dc1f-40fb-a67d-5fbebc7deec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:55.713060 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c426b-dc1f-40fb-a67d-5fbebc7deec6 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:55.713557 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c426b-dc1f-40fb-a67d-5fbebc7deec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:55.714014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c426b-dc1f-40fb-a67d-5fbebc7deec6 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:55.714941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c426b-dc1f-40fb-a67d-5fbebc7deec6 None admin] RBAC: Adding query filter params (name=network) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:55.715771 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c426b-dc1f-40fb-a67d-5fbebc7deec6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:55.717388 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c426b-dc1f-40fb-a67d-5fbebc7deec6 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:55.724356 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 413/824] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 17:21:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:55.734234 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c426b-dc1f-40fb-a67d-5fbebc7deec6 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:55.734410 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 412/825] 10.222.0.40 () {56 vars in 1149 bytes} [Thu Jun 16 17:21:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:55.736300 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bf131-69bb-4f84-b5fb-d5c329ece5d4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:55.764982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:55.879423 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c426b-dc1f-40fb-a67d-5fbebc7deec6 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:55.881259 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bf131-69bb-4f84-b5fb-d5c329ece5d4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:55.940143 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d88bdf-9ded-4186-96fb-da2f7da296a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:55.941289 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199fca1-7e44-470e-b024-6c2be2b151ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:55.941584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d88bdf-9ded-4186-96fb-da2f7da296a3 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:55.941965 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199fca1-7e44-470e-b024-6c2be2b151ba None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:55.942253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199fca1-7e44-470e-b024-6c2be2b151ba None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:55.942536 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199fca1-7e44-470e-b024-6c2be2b151ba None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:55.942744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d88bdf-9ded-4186-96fb-da2f7da296a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:55.943220 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d88bdf-9ded-4186-96fb-da2f7da296a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:55.943321 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199fca1-7e44-470e-b024-6c2be2b151ba None admin] RBAC: Adding query filter params (type=network) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:55.943561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199fca1-7e44-470e-b024-6c2be2b151ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:55.945134 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199fca1-7e44-470e-b024-6c2be2b151ba None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:55.952905 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 414/826] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 17:21:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:55.958294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199fca1-7e44-470e-b024-6c2be2b151ba None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:55.960928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d88bdf-9ded-4186-96fb-da2f7da296a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:55.962432 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d88bdf-9ded-4186-96fb-da2f7da296a3 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:55.984555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-f1d88bdf-9ded-4186-96fb-da2f7da296a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:55.984555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:55.984555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:55.984555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:55.984555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:55.984555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:55.984555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:55.984555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:55.984555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:55.984555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:55.984555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:55.984555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:55.984555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:55.984555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:55.984555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:55.986086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:55.986086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:55.986086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:55.986086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:55.986086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:55.986086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:55.986086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:55.986086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:55.986086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:55.986086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:55.986086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:55.986086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:55.986086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:55.986086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:55.986086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:55.986086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:55.986086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:55.987102 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:55.987102 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:55.987102 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:55.987102 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:55.989805 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 413/827] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:55.991811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d88bdf-9ded-4186-96fb-da2f7da296a3 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:56.065177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:56.092698 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:56.135122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199fca1-7e44-470e-b024-6c2be2b151ba None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:56.151696 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d88bdf-9ded-4186-96fb-da2f7da296a3 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:56.206279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8ae74-6b67-41fa-a624-76fe63adc994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:56.208464 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8ae74-6b67-41fa-a624-76fe63adc994 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:56.208464 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8ae74-6b67-41fa-a624-76fe63adc994 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:56.208464 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8ae74-6b67-41fa-a624-76fe63adc994 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:56.208464 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad8ae74-6b67-41fa-a624-76fe63adc994 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:56.208464 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad8ae74-6b67-41fa-a624-76fe63adc994 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:56.210097 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad8ae74-6b67-41fa-a624-76fe63adc994 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:56.226712 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb692e-f2a3-44c9-b2b6-5a0317ae6f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:56.227404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb692e-f2a3-44c9-b2b6-5a0317ae6f1b None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:56.227697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb692e-f2a3-44c9-b2b6-5a0317ae6f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:56.227982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb692e-f2a3-44c9-b2b6-5a0317ae6f1b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:56.228744 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb692e-f2a3-44c9-b2b6-5a0317ae6f1b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:56.229218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 414/828] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:56.230613 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8ae74-6b67-41fa-a624-76fe63adc994 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:56.243240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb692e-f2a3-44c9-b2b6-5a0317ae6f1b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:56.286363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 415/829] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jun 16 17:21:55 2022] POST /identity/v3/endpoints => generated 327 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:21:56.369782 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:56.382755 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8ae74-6b67-41fa-a624-76fe63adc994 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:56.390254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8f103-2185-484f-9ffd-9d1ba564c500 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:56.390481 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8f103-2185-484f-9ffd-9d1ba564c500 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:56.390691 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8f103-2185-484f-9ffd-9d1ba564c500 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:56.392331 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 416/830] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:21:56.398068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fceca-0b1f-427d-9a75-bea1802c5fda None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:56.398355 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fceca-0b1f-427d-9a75-bea1802c5fda None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:56.398646 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fceca-0b1f-427d-9a75-bea1802c5fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:56.472695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee4097-f77b-4bcf-881b-755aaab717fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:56.473507 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee4097-f77b-4bcf-881b-755aaab717fd None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:56.473804 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee4097-f77b-4bcf-881b-755aaab717fd None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:56.474094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee4097-f77b-4bcf-881b-755aaab717fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:56.498652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee4097-f77b-4bcf-881b-755aaab717fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:56.500594 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee4097-f77b-4bcf-881b-755aaab717fd None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:56.523798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-6aee4097-f77b-4bcf-881b-755aaab717fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:56.523798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:56.523798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:56.523798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:56.523798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:56.523798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:56.523798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:56.523798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:56.523798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:56.523798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:56.523798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:56.523798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:56.523798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:56.523798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:56.523798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:56.524855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:56.524855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:56.524855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:56.524855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:56.524855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:56.524855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:56.524855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:56.524855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:56.524855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:56.524855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:56.524855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:56.524855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:56.524855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:56.524855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:56.524855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:56.524855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:56.524855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:56.525813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:56.525813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:56.525813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:56.525813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:56.525813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:56.525813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:56.525813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:56.525813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:56.525813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 415/831] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:56.539045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee4097-f77b-4bcf-881b-755aaab717fd None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:56.560354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-932fceca-0b1f-427d-9a75-bea1802c5fda None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:56.561646 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-932fceca-0b1f-427d-9a75-bea1802c5fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:56.638124 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 417/832] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:21:56.661526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fceca-0b1f-427d-9a75-bea1802c5fda None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:56.670244 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee4097-f77b-4bcf-881b-755aaab717fd None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:56.722258 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9395f-61c4-4cfe-9850-b32fd5b051ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:56.722992 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9395f-61c4-4cfe-9850-b32fd5b051ea None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:56.723510 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9395f-61c4-4cfe-9850-b32fd5b051ea None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:56.723847 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9395f-61c4-4cfe-9850-b32fd5b051ea None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:56.724842 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf9395f-61c4-4cfe-9850-b32fd5b051ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:56.725950 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf9395f-61c4-4cfe-9850-b32fd5b051ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:56.727701 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf9395f-61c4-4cfe-9850-b32fd5b051ea None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:56.747096 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 416/833] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:56.753512 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:56.754586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9395f-61c4-4cfe-9850-b32fd5b051ea None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:56.772357 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:56.787765 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fceca-0b1f-427d-9a75-bea1802c5fda None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:56.831172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c43e6-b5e9-4dac-bb50-e544218f5034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:56.832352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c43e6-b5e9-4dac-bb50-e544218f5034 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:56.832794 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c43e6-b5e9-4dac-bb50-e544218f5034 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:56.833084 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c43e6-b5e9-4dac-bb50-e544218f5034 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:56.847996 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c43e6-b5e9-4dac-bb50-e544218f5034 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:56.858260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-d56c43e6-b5e9-4dac-bb50-e544218f5034 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:21:56.858260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:56.858260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:56.858260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:56.858260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:56.858260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:56.858260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:56.858260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:56.858260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:56.858260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:56.858260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:56.858260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:56.858260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:56.858260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:56.858260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:56.858260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:56.859231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:56.859231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:56.859231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:56.859231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:56.859231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:56.859231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:56.859231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:56.859231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:56.859231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:56.859231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:56.859231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:56.859231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:56.859231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:56.859231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:56.859231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:56.859231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:56.860291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:56.860291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:56.860291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:56.860291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:56.860291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:56.860291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:56.860291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:56.860291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:56.860291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:56.860291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:56.860291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:21:56.860291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:56.860291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 418/834] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:21:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:56.867319 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c43e6-b5e9-4dac-bb50-e544218f5034 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:56.878922 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9395f-61c4-4cfe-9850-b32fd5b051ea None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:56.919753 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5936f-07ac-4183-9579-af5ad20a30a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:56.920754 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5936f-07ac-4183-9579-af5ad20a30a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:56.921160 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5936f-07ac-4183-9579-af5ad20a30a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:56.921524 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5936f-07ac-4183-9579-af5ad20a30a8 None admin] PATH_INFO: `/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/cfc2f31f654e486d8505d80aad17d17c/roles/0e34a6f9b89a4e29951fb189abacb34a` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:56.937570 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5936f-07ac-4183-9579-af5ad20a30a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=35a535a305e740368b7a4d77d0ad22a0, user_id=cfc2f31f654e486d8505d80aad17d17c, role_id=0e34a6f9b89a4e29951fb189abacb34a)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:56.939165 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5936f-07ac-4183-9579-af5ad20a30a8 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:56.956251 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 417/835] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:21:56 2022] PUT /identity/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/cfc2f31f654e486d8505d80aad17d17c/roles/0e34a6f9b89a4e29951fb189abacb34a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:21:57.011760 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c43e6-b5e9-4dac-bb50-e544218f5034 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:57.053351 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb246bb-0814-4662-a6e4-a84f28ab78b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:57.054137 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb246bb-0814-4662-a6e4-a84f28ab78b9 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:57.054432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb246bb-0814-4662-a6e4-a84f28ab78b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:57.054768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb246bb-0814-4662-a6e4-a84f28ab78b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:57.055564 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb246bb-0814-4662-a6e4-a84f28ab78b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:57.055882 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb246bb-0814-4662-a6e4-a84f28ab78b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:57.057310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb246bb-0814-4662-a6e4-a84f28ab78b9 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:57.059055 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:57.069862 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 419/836] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:21:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:57.076382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5936f-07ac-4183-9579-af5ad20a30a8 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:57.142423 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:57.196239 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5936f-07ac-4183-9579-af5ad20a30a8 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:57.232241 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801c4f4-ee59-47c8-9c6d-4d520d7c98bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:57.233056 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801c4f4-ee59-47c8-9c6d-4d520d7c98bd None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:57.233372 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801c4f4-ee59-47c8-9c6d-4d520d7c98bd None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:57.233702 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801c4f4-ee59-47c8-9c6d-4d520d7c98bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:57.245754 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801c4f4-ee59-47c8-9c6d-4d520d7c98bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:57.247264 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801c4f4-ee59-47c8-9c6d-4d520d7c98bd None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:57.267442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-c801c4f4-ee59-47c8-9c6d-4d520d7c98bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:57.267442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:57.267442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:57.267442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:57.267442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:57.267442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:57.267442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:57.267442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:57.267442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:57.267442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:57.267442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:57.267442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:57.267442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:57.267442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:57.267442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:57.268358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:57.268358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:57.268358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:57.268358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:57.268358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:57.268358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:57.268358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:57.268358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:57.268358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:57.268358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:57.268358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:57.268358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:57.268358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:57.268358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:57.268358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:57.268358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:57.268358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:57.269309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:57.269309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:57.269309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:57.269309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:57.269309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 418/837] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:57.274216 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb246bb-0814-4662-a6e4-a84f28ab78b9 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:57.380692 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb246bb-0814-4662-a6e4-a84f28ab78b9 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:57.417442 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9609e2-7efd-4806-a70d-db5907ba8e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:57.418412 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9609e2-7efd-4806-a70d-db5907ba8e81 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:57.418911 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9609e2-7efd-4806-a70d-db5907ba8e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:57.419374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9609e2-7efd-4806-a70d-db5907ba8e81 None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:57.420281 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9609e2-7efd-4806-a70d-db5907ba8e81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:57.420774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9609e2-7efd-4806-a70d-db5907ba8e81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:57.422532 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9609e2-7efd-4806-a70d-db5907ba8e81 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:57.435410 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 420/838] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:57.441177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801c4f4-ee59-47c8-9c6d-4d520d7c98bd None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:57.550935 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801c4f4-ee59-47c8-9c6d-4d520d7c98bd None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:57.562760 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:57.588284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f463d6c-38ec-4d4c-a578-b3fc80e33da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:57.589090 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f463d6c-38ec-4d4c-a578-b3fc80e33da0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:57.589406 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f463d6c-38ec-4d4c-a578-b3fc80e33da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:57.589688 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f463d6c-38ec-4d4c-a578-b3fc80e33da0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:57.597581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f463d6c-38ec-4d4c-a578-b3fc80e33da0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:57.599033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f463d6c-38ec-4d4c-a578-b3fc80e33da0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:57.614569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-1f463d6c-38ec-4d4c-a578-b3fc80e33da0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:21:57.614569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:57.614569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:57.614569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:57.614569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:57.614569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:57.614569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:57.614569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:57.614569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:57.614569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:57.614569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:57.614569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:57.614569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:21:57.614569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:21:57.614569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:21:57.614569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:21:57.615554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:57.615554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:57.615554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:21:57.615554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:57.615554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:21:57.615554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:21:57.615554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:57.615554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:57.615554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:57.615554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:57.615554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:57.615554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:57.615554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:57.615554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:57.615554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:57.615554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:57.615554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:57.615554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:57.616532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:57.616532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:57.616532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:21:57.616532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:21:57.616532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:21:57.616532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:21:57.616532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:21:57.616532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:21:57.616532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:21:57.616532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:57.616532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 419/839] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:21:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:57.621692 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9609e2-7efd-4806-a70d-db5907ba8e81 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:57.685444 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:57.736976 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9609e2-7efd-4806-a70d-db5907ba8e81 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:57.777337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b123948c-3d47-467a-9f04-e731812f35e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:57.778387 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123948c-3d47-467a-9f04-e731812f35e2 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:57.778880 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123948c-3d47-467a-9f04-e731812f35e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:57.779386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123948c-3d47-467a-9f04-e731812f35e2 None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:57.780479 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b123948c-3d47-467a-9f04-e731812f35e2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:57.781542 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b123948c-3d47-467a-9f04-e731812f35e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:57.783196 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b123948c-3d47-467a-9f04-e731812f35e2 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:57.802391 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 421/840] 10.222.0.40 () {56 vars in 1135 bytes} [Thu Jun 16 17:21:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:57.813183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f463d6c-38ec-4d4c-a578-b3fc80e33da0 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:57.919241 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f463d6c-38ec-4d4c-a578-b3fc80e33da0 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:57.956900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65be5a4-1256-4709-b02e-f538e9827922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:57.957697 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65be5a4-1256-4709-b02e-f538e9827922 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:57.957989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65be5a4-1256-4709-b02e-f538e9827922 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:57.958270 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65be5a4-1256-4709-b02e-f538e9827922 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:58.004602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:58.027511 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aba09c-0b29-484a-bb96-4589ff7d57b0 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:58.028127 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aba09c-0b29-484a-bb96-4589ff7d57b0 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:58.028641 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aba09c-0b29-484a-bb96-4589ff7d57b0 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:58.031169 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 422/841] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:58.036409 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487fda1-03e7-484a-a6ef-506f50bbe41c None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:58.036944 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487fda1-03e7-484a-a6ef-506f50bbe41c None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:58.037480 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487fda1-03e7-484a-a6ef-506f50bbe41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:58.040325 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65be5a4-1256-4709-b02e-f538e9827922 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:58.041896 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65be5a4-1256-4709-b02e-f538e9827922 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:58.055471 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-0487fda1-03e7-484a-a6ef-506f50bbe41c None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:58.056956 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0487fda1-03e7-484a-a6ef-506f50bbe41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:58.137480 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 423/842] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:21:58.154541 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487fda1-03e7-484a-a6ef-506f50bbe41c None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:58.157858 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 420/843] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:21:57 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:21:58.166285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65be5a4-1256-4709-b02e-f538e9827922 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:58.210184 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:58.260484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:58.281480 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487fda1-03e7-484a-a6ef-506f50bbe41c None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:58.295305 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65be5a4-1256-4709-b02e-f538e9827922 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:58.317676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a550e-3f77-4b48-acfb-e6e0246b48d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:58.318739 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a550e-3f77-4b48-acfb-e6e0246b48d2 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:58.319023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a550e-3f77-4b48-acfb-e6e0246b48d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:58.319303 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a550e-3f77-4b48-acfb-e6e0246b48d2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:58.319960 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a550e-3f77-4b48-acfb-e6e0246b48d2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:58.321330 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a550e-3f77-4b48-acfb-e6e0246b48d2 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:58.331294 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780bcc9-f60f-4f43-aa40-ab6ade32281c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:58.332096 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780bcc9-f60f-4f43-aa40-ab6ade32281c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:58.332388 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780bcc9-f60f-4f43-aa40-ab6ade32281c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:58.332671 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780bcc9-f60f-4f43-aa40-ab6ade32281c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:58.333887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-db7a550e-3f77-4b48-acfb-e6e0246b48d2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:21:58.333887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:58.333887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:21:58.333887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:21:58.333887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:21:58.333887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:21:58.333887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:21:58.333887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:21:58.333887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:21:58.333887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:58.333887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:21:58.333887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:58.333887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:58.333887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:58.333887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:58.333887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:58.333887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:58.334873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:58.334873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:58.334873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:58.334873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:58.334873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:58.334873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:58.334873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:21:58.334873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:21:58.334873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:21:58.334873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:21:58.334873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:58.334873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:58.334873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:58.334873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:58.334873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:58.334873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:58.335756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:58.335756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:58.335756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:58.335756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:58.335756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:58.335756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:58.335756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:58.335756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:58.335756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:58.335756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:58.335756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:21:58.335756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:21:58.335756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:21:58.335756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:58.335756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 424/844] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:21:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:58.337415 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ea636-12fa-4386-a5c8-540041ea939c None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:58.337415 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ea636-12fa-4386-a5c8-540041ea939c None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:58.337415 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ea636-12fa-4386-a5c8-540041ea939c None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:58.338316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 425/845] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:58.342700 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ea636-12fa-4386-a5c8-540041ea939c None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:58.349812 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780bcc9-f60f-4f43-aa40-ab6ade32281c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:58.351279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780bcc9-f60f-4f43-aa40-ab6ade32281c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:58.369226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-b780bcc9-f60f-4f43-aa40-ab6ade32281c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:58.369226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:58.369226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:58.369226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:58.369226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:58.369226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:58.369226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:58.369226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:58.369226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:58.369226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:58.369226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:58.369226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:58.369226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:58.369226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:58.369226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:58.370171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:58.370171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:58.370171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:58.370171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:58.370171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:58.370171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:58.370171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:58.370171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:58.370171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:58.370171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:58.370171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:58.370171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:58.370171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:58.370171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:58.370171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:58.370171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:58.370171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:58.371269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:58.371269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:58.371269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:58.371269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:21:58.371269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 421/846] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:58.372299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c42a85-553c-436b-ac2f-8a7541958ce2 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:58.372299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c42a85-553c-436b-ac2f-8a7541958ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:58.372299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c42a85-553c-436b-ac2f-8a7541958ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:58.398775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-88c42a85-553c-436b-ac2f-8a7541958ce2 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:58.399975 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-88c42a85-553c-436b-ac2f-8a7541958ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:58.458675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ea636-12fa-4386-a5c8-540041ea939c None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:58.468378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:58.485170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 422/847] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:21:58.485170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c42a85-553c-436b-ac2f-8a7541958ce2 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:58.509737 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968c9c1-7539-41bf-b2e0-e3e936ba99c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:58.510452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968c9c1-7539-41bf-b2e0-e3e936ba99c4 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:58.510741 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968c9c1-7539-41bf-b2e0-e3e936ba99c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:58.511209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968c9c1-7539-41bf-b2e0-e3e936ba99c4 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:58.511928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968c9c1-7539-41bf-b2e0-e3e936ba99c4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:58.512236 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968c9c1-7539-41bf-b2e0-e3e936ba99c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:58.513634 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968c9c1-7539-41bf-b2e0-e3e936ba99c4 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:58.523276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 426/848] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:21:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:58.523276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968c9c1-7539-41bf-b2e0-e3e936ba99c4 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:58.550554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:58.603058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c42a85-553c-436b-ac2f-8a7541958ce2 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:58.632644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968c9c1-7539-41bf-b2e0-e3e936ba99c4 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:58.642476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06112663-46c4-458d-b1e2-f82a0980f5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:58.643597 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06112663-46c4-458d-b1e2-f82a0980f5c0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:58.643888 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06112663-46c4-458d-b1e2-f82a0980f5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:58.644573 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06112663-46c4-458d-b1e2-f82a0980f5c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:58.645635 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06112663-46c4-458d-b1e2-f82a0980f5c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:58.645951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06112663-46c4-458d-b1e2-f82a0980f5c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:58.647997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06112663-46c4-458d-b1e2-f82a0980f5c0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:58.660695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 423/849] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:58.661440 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06112663-46c4-458d-b1e2-f82a0980f5c0 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:58.673811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21860d02-3abf-4a52-80a1-39c7ef568486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:58.674631 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21860d02-3abf-4a52-80a1-39c7ef568486 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:58.674911 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21860d02-3abf-4a52-80a1-39c7ef568486 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:58.675193 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21860d02-3abf-4a52-80a1-39c7ef568486 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:58.675857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21860d02-3abf-4a52-80a1-39c7ef568486 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:58.677244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21860d02-3abf-4a52-80a1-39c7ef568486 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:58.690041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-21860d02-3abf-4a52-80a1-39c7ef568486 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:21:58.690041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:58.690041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:21:58.690041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:21:58.690041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:21:58.690041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:21:58.690041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:21:58.690041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:21:58.690041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:21:58.690041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:58.690041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:21:58.690041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:58.690041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:58.690041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:58.690041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:58.690041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:58.691510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:58.691510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:58.691510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:58.691510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:58.691510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:58.691510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:58.691510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:58.691510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:21:58.691510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:21:58.691510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:21:58.691510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:21:58.691510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:58.691510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:58.691510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:58.691510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:58.691510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:58.692432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:58.692432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:58.692432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:58.692432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:58.692432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:58.692432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:58.692432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:58.692432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:58.692432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:58.692432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:58.692432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:58.692432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:21:58.692432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:21:58.692432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:21:58.692432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:58.692432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 427/850] 10.222.0.40 () {56 vars in 1132 bytes} [Thu Jun 16 17:21:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:58.693360 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21860d02-3abf-4a52-80a1-39c7ef568486 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:58.732461 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:58.776747 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06112663-46c4-458d-b1e2-f82a0980f5c0 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:58.790681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:58.806455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21860d02-3abf-4a52-80a1-39c7ef568486 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:58.813344 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7059965-bfc4-44e5-a4c7-bea94a2f6cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:58.814028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7059965-bfc4-44e5-a4c7-bea94a2f6cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:58.814680 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7059965-bfc4-44e5-a4c7-bea94a2f6cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:58.814680 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7059965-bfc4-44e5-a4c7-bea94a2f6cd5 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:58.815436 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7059965-bfc4-44e5-a4c7-bea94a2f6cd5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:58.815751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7059965-bfc4-44e5-a4c7-bea94a2f6cd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:58.817123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7059965-bfc4-44e5-a4c7-bea94a2f6cd5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:58.821947 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 424/851] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:21:58 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:58.822642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7059965-bfc4-44e5-a4c7-bea94a2f6cd5 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:58.861225 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a2837-7a15-4d50-af02-fe21f80740c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:58.862703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a2837-7a15-4d50-af02-fe21f80740c0 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:58.863214 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a2837-7a15-4d50-af02-fe21f80740c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:58.863799 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a2837-7a15-4d50-af02-fe21f80740c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:58.875247 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a2837-7a15-4d50-af02-fe21f80740c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:58.877639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a2837-7a15-4d50-af02-fe21f80740c0 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:58.900858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-470a2837-7a15-4d50-af02-fe21f80740c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:58.900858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:58.900858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:58.900858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:58.900858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:58.900858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:58.900858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:58.900858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:58.900858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:58.900858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:58.900858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:58.900858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:58.900858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:21:58.900858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:21:58.900858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:21:58.901870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:21:58.901870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:58.901870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:58.901870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:58.901870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:58.901870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:58.901870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:58.901870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:58.901870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:58.901870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:58.901870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:58.901870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:58.901870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:58.901870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:58.901870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:58.901870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:58.901870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:58.902774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:21:58.902774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:21:58.902774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:21:58.902774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:21:58.902774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:21:58.902774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:21:58.902774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:21:58.902774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:58.905891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 428/852] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:21:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:58.911507 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a2837-7a15-4d50-af02-fe21f80740c0 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:58.962673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7059965-bfc4-44e5-a4c7-bea94a2f6cd5 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:59.006260 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41974115-6d98-4c14-969c-9639afd99995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:59.007080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41974115-6d98-4c14-969c-9639afd99995 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:59.007471 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41974115-6d98-4c14-969c-9639afd99995 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:59.008119 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41974115-6d98-4c14-969c-9639afd99995 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:59.008941 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41974115-6d98-4c14-969c-9639afd99995 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:59.009492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41974115-6d98-4c14-969c-9639afd99995 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:59.011048 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41974115-6d98-4c14-969c-9639afd99995 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:59.018357 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 425/853] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 17:21:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:59.023425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41974115-6d98-4c14-969c-9639afd99995 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:59.041534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:59.046863 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a2837-7a15-4d50-af02-fe21f80740c0 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:59.077903 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:59.085220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc14645-10c7-4652-aa9d-813c1db6dfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:59.085896 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc14645-10c7-4652-aa9d-813c1db6dfce None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:59.086248 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc14645-10c7-4652-aa9d-813c1db6dfce None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:59.086507 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc14645-10c7-4652-aa9d-813c1db6dfce None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:59.087518 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc14645-10c7-4652-aa9d-813c1db6dfce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:59.087836 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc14645-10c7-4652-aa9d-813c1db6dfce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:59.089468 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc14645-10c7-4652-aa9d-813c1db6dfce None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:59.103170 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 429/854] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:21:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:59.109761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc14645-10c7-4652-aa9d-813c1db6dfce None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:59.164526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41974115-6d98-4c14-969c-9639afd99995 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:59.211909 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3af89-5c2b-4a53-8dcc-4eeaa9508338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:59.212928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3af89-5c2b-4a53-8dcc-4eeaa9508338 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:59.213444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3af89-5c2b-4a53-8dcc-4eeaa9508338 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:59.213931 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3af89-5c2b-4a53-8dcc-4eeaa9508338 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:59.215647 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3af89-5c2b-4a53-8dcc-4eeaa9508338 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:59.216448 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3af89-5c2b-4a53-8dcc-4eeaa9508338 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:59.218519 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3af89-5c2b-4a53-8dcc-4eeaa9508338 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:59.234529 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 426/855] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 17:21:59 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:21:59.247388 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc14645-10c7-4652-aa9d-813c1db6dfce None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:59.282819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673d72e-a435-4a4c-aee2-517b5a429c38 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:59.283252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673d72e-a435-4a4c-aee2-517b5a429c38 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:59.283493 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673d72e-a435-4a4c-aee2-517b5a429c38 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:59.284860 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 427/856] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:21:59.290977 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989021b0-0a4b-483a-97d8-6e78faa1c36d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:59.291491 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989021b0-0a4b-483a-97d8-6e78faa1c36d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:59.291987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989021b0-0a4b-483a-97d8-6e78faa1c36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:59.294048 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50483ae5-8604-4827-b26f-5441d820e9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:59.294743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50483ae5-8604-4827-b26f-5441d820e9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:59.295042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50483ae5-8604-4827-b26f-5441d820e9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:59.295329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50483ae5-8604-4827-b26f-5441d820e9d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:59.296288 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50483ae5-8604-4827-b26f-5441d820e9d6 None admin] RBAC: Adding query filter params (role.id=25a16d8ea4084e9eacdebb7a4dcd2337, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=b8a37d1201d54afdad9b228ef4f2d787) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:59.296663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50483ae5-8604-4827-b26f-5441d820e9d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:59.298276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50483ae5-8604-4827-b26f-5441d820e9d6 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:59.309212 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 430/857] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:21:59 2022] GET /identity/v3/role_assignments?user.id=b8a37d1201d54afdad9b228ef4f2d787&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=25a16d8ea4084e9eacdebb7a4dcd2337 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:59.317720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-989021b0-0a4b-483a-97d8-6e78faa1c36d None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:21:59.319913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-989021b0-0a4b-483a-97d8-6e78faa1c36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:21:59.425451 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 428/858] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:21:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:21:59.449828 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50483ae5-8604-4827-b26f-5441d820e9d6 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:59.478123 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:59.569921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50483ae5-8604-4827-b26f-5441d820e9d6 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:59.606099 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6b055-ef49-4e3b-a7ec-040caa3d600b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:59.606905 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6b055-ef49-4e3b-a7ec-040caa3d600b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:59.607190 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6b055-ef49-4e3b-a7ec-040caa3d600b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:59.607463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6b055-ef49-4e3b-a7ec-040caa3d600b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:59.618633 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6b055-ef49-4e3b-a7ec-040caa3d600b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:21:59.626946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-d5f6b055-ef49-4e3b-a7ec-040caa3d600b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:59.626946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:59.626946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:59.626946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:59.626946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:59.626946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:59.626946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:59.626946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:59.626946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:59.626946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:59.626946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:59.626946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:59.626946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:21:59.626946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:21:59.626946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:21:59.626946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:21:59.628113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:21:59.628113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:21:59.628113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:21:59.628113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:21:59.628113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:59.628113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:59.628113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:59.628113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:59.628113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:59.628113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:59.628113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:59.628113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:59.628113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:59.628113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:59.628113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:59.628113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:59.629429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:59.629429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:59.629429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:59.629429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:59.629429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:21:59.629429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:21:59.629429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:21:59.629429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:21:59.629429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:21:59.629429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:21:59.629429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:21:59.629429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:59.629429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 431/859] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:21:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:59.634256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989021b0-0a4b-483a-97d8-6e78faa1c36d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:59.673955 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:59.745224 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989021b0-0a4b-483a-97d8-6e78faa1c36d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:59.782115 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac98e00-fbfc-4ca2-9abf-95f0533284f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:59.782806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac98e00-fbfc-4ca2-9abf-95f0533284f4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:59.783104 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac98e00-fbfc-4ca2-9abf-95f0533284f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:59.783394 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac98e00-fbfc-4ca2-9abf-95f0533284f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:59.784163 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac98e00-fbfc-4ca2-9abf-95f0533284f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:21:59.784483 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac98e00-fbfc-4ca2-9abf-95f0533284f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:59.785908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac98e00-fbfc-4ca2-9abf-95f0533284f4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:59.795293 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 429/860] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 17:21:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:21:59.801007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6b055-ef49-4e3b-a7ec-040caa3d600b None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:21:59.888130 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:59.911239 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6b055-ef49-4e3b-a7ec-040caa3d600b None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:21:59.945545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45936823-7e6c-4b12-956a-6bbf5c04b417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:21:59.946389 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45936823-7e6c-4b12-956a-6bbf5c04b417 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:21:59.946680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45936823-7e6c-4b12-956a-6bbf5c04b417 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:21:59.946960 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45936823-7e6c-4b12-956a-6bbf5c04b417 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:21:59.958644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45936823-7e6c-4b12-956a-6bbf5c04b417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:21:59.960149 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45936823-7e6c-4b12-956a-6bbf5c04b417 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:21:59.978267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-45936823-7e6c-4b12-956a-6bbf5c04b417 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:59.978267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:21:59.978267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:21:59.978267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:21:59.978267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:21:59.978267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:21:59.978267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:21:59.978267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:21:59.978267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:21:59.978267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:21:59.978267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:21:59.978267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:21:59.978267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:21:59.978267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:21:59.978267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:21:59.979259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:21:59.979259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:21:59.979259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:21:59.979259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:21:59.979259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:21:59.979259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:21:59.979259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:21:59.979259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:21:59.979259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:21:59.979259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:21:59.979259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:21:59.979259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:21:59.979259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:21:59.979259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:21:59.979259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:21:59.979259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:21:59.979259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:21:59.980199 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:21:59.980199 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:21:59.980199 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:21:59.980199 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:21:59.980199 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 432/861] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:21:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:21:59.986769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac98e00-fbfc-4ca2-9abf-95f0533284f4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:00.091226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:00.099436 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac98e00-fbfc-4ca2-9abf-95f0533284f4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:00.135595 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b065b36-c754-4a88-a2e1-8d909a37c1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:00.136324 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b065b36-c754-4a88-a2e1-8d909a37c1b9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:00.136613 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b065b36-c754-4a88-a2e1-8d909a37c1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:00.136895 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b065b36-c754-4a88-a2e1-8d909a37c1b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:00.137666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b065b36-c754-4a88-a2e1-8d909a37c1b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:00.137983 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b065b36-c754-4a88-a2e1-8d909a37c1b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:00.139354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b065b36-c754-4a88-a2e1-8d909a37c1b9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:00.152084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 430/862] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:00.158073 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45936823-7e6c-4b12-956a-6bbf5c04b417 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:00.269299 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45936823-7e6c-4b12-956a-6bbf5c04b417 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:00.300452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:00.306329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac053f4-3054-4c0d-ab05-d9f89c36d908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:00.307162 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac053f4-3054-4c0d-ab05-d9f89c36d908 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:00.307455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac053f4-3054-4c0d-ab05-d9f89c36d908 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:00.307742 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac053f4-3054-4c0d-ab05-d9f89c36d908 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:00.316759 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac053f4-3054-4c0d-ab05-d9f89c36d908 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:00.318294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac053f4-3054-4c0d-ab05-d9f89c36d908 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:00.333844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-5ac053f4-3054-4c0d-ab05-d9f89c36d908 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:22:00.333844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:00.333844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:00.333844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:00.333844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:00.333844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:00.333844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:00.333844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:00.333844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:00.333844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:00.333844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:00.333844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:00.333844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:22:00.333844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:22:00.333844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:22:00.333844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:22:00.334818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:00.334818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:00.334818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:22:00.334818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:22:00.334818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:22:00.334818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:22:00.334818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:00.334818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:00.334818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:00.334818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:00.334818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:00.334818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:00.334818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:00.334818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:00.334818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:00.334818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:00.334818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:00.334818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:00.335796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:00.335796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:00.335796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:22:00.335796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:22:00.335796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:22:00.335796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:22:00.335796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:22:00.335796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:22:00.335796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:22:00.335796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:00.335796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 433/863] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jun 16 17:22:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:00.342878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b065b36-c754-4a88-a2e1-8d909a37c1b9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:00.451468 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b065b36-c754-4a88-a2e1-8d909a37c1b9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:00.489637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6825c06-127f-4db4-a87e-4d693cfa5ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:00.490475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6825c06-127f-4db4-a87e-4d693cfa5ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:00.490769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6825c06-127f-4db4-a87e-4d693cfa5ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:00.491054 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6825c06-127f-4db4-a87e-4d693cfa5ff2 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:00.491911 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6825c06-127f-4db4-a87e-4d693cfa5ff2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:00.492229 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6825c06-127f-4db4-a87e-4d693cfa5ff2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:00.493705 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6825c06-127f-4db4-a87e-4d693cfa5ff2 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:00.513073 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 431/864] 10.222.0.40 () {56 vars in 1149 bytes} [Thu Jun 16 17:22:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:00.522227 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac053f4-3054-4c0d-ab05-d9f89c36d908 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:00.625464 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac053f4-3054-4c0d-ab05-d9f89c36d908 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:00.662749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b16bd8-0204-485e-8376-61cce663f0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:00.663811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b16bd8-0204-485e-8376-61cce663f0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:00.664304 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b16bd8-0204-485e-8376-61cce663f0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:00.664797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b16bd8-0204-485e-8376-61cce663f0f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:00.727432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:00.738654 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b16bd8-0204-485e-8376-61cce663f0f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:00.740380 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b16bd8-0204-485e-8376-61cce663f0f4 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:00.844385 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 434/865] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:22:00 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:22:00.857237 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6825c06-127f-4db4-a87e-4d693cfa5ff2 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:00.876241 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:00.958582 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6825c06-127f-4db4-a87e-4d693cfa5ff2 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:00.993638 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069095a-856b-4fef-8bc8-99eee57a6570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:00.994438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069095a-856b-4fef-8bc8-99eee57a6570 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:00.994735 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069095a-856b-4fef-8bc8-99eee57a6570 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:00.995020 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069095a-856b-4fef-8bc8-99eee57a6570 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:01.006481 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069095a-856b-4fef-8bc8-99eee57a6570 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:01.008216 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069095a-856b-4fef-8bc8-99eee57a6570 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:01.027790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-6069095a-856b-4fef-8bc8-99eee57a6570 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:01.027790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:01.027790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:01.027790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:01.027790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:01.027790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:01.027790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:01.027790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:01.027790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:01.027790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:01.027790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:01.027790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:01.027790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:22:01.027790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:22:01.027790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:22:01.028784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:22:01.028784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:01.028784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:01.028784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:01.028784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:01.028784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:01.028784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:01.028784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:01.028784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:01.028784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:01.028784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:01.028784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:01.028784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:01.028784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:01.028784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:01.028784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:01.028784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:01.029742 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:22:01.029742 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:22:01.029742 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:01.029742 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:01.033628 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 432/866] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:01.035950 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b16bd8-0204-485e-8376-61cce663f0f4 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:01.092360 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9d692-c3a8-43f7-9e98-3417ab874d46 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:01.092684 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9d692-c3a8-43f7-9e98-3417ab874d46 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:01.093013 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9d692-c3a8-43f7-9e98-3417ab874d46 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:01.094566 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 433/867] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:22:01.099868 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745f3ba-8257-4774-bfe2-a168b466bb01 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:01.100164 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745f3ba-8257-4774-bfe2-a168b466bb01 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:01.100448 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745f3ba-8257-4774-bfe2-a168b466bb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:01.128122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a745f3ba-8257-4774-bfe2-a168b466bb01 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:01.129429 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a745f3ba-8257-4774-bfe2-a168b466bb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:01.145665 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:01.155376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b16bd8-0204-485e-8376-61cce663f0f4 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:01.194805 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebbce8-fad5-4801-b800-65f5079ddea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:01.196046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebbce8-fad5-4801-b800-65f5079ddea8 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:01.196548 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebbce8-fad5-4801-b800-65f5079ddea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:01.197090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebbce8-fad5-4801-b800-65f5079ddea8 None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:01.198227 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ebbce8-fad5-4801-b800-65f5079ddea8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:01.198777 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ebbce8-fad5-4801-b800-65f5079ddea8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:01.200566 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ebbce8-fad5-4801-b800-65f5079ddea8 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:01.210857 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:01.219392 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 435/868] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:01.220555 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebbce8-fad5-4801-b800-65f5079ddea8 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:01.220945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 434/869] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:01.247221 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745f3ba-8257-4774-bfe2-a168b466bb01 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:01.327683 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebbce8-fad5-4801-b800-65f5079ddea8 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:01.362728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745f3ba-8257-4774-bfe2-a168b466bb01 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:01.364400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8f226-f1b7-4521-97c4-fbb7a17adb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:01.366303 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8f226-f1b7-4521-97c4-fbb7a17adb3d None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:01.366816 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8f226-f1b7-4521-97c4-fbb7a17adb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:01.367309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8f226-f1b7-4521-97c4-fbb7a17adb3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:01.376628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb8f226-f1b7-4521-97c4-fbb7a17adb3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:01.378644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb8f226-f1b7-4521-97c4-fbb7a17adb3d None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:01.397206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-3bb8f226-f1b7-4521-97c4-fbb7a17adb3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:22:01.397206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:01.397206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:01.397206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:01.397206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:01.397206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:01.397206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:01.397206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:01.397206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:01.397206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:01.397206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:01.397206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:01.397206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:22:01.397206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:22:01.397206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:22:01.398283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:22:01.398283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:01.398283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:01.398283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:01.398283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:01.398283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:01.398283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:01.398283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:01.398283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:01.398283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:01.398283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:01.398283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:01.398283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:01.398283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:01.398283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:01.398283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:01.398283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:01.399290 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:22:01.399290 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:22:01.399290 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:22:01.399290 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:22:01.399290 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:22:01.399290 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:22:01.399290 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:22:01.399290 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:01.404227 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 436/870] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:22:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:01.406186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8f226-f1b7-4521-97c4-fbb7a17adb3d None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:01.413013 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc8ece-9adc-4230-bace-57efbaf1cee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:01.414309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc8ece-9adc-4230-bace-57efbaf1cee1 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:01.414810 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc8ece-9adc-4230-bace-57efbaf1cee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:01.415280 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc8ece-9adc-4230-bace-57efbaf1cee1 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:01.416312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc8ece-9adc-4230-bace-57efbaf1cee1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:01.418718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc8ece-9adc-4230-bace-57efbaf1cee1 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:01.425570 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:01.442639 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 435/871] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 17:22:01 2022] POST /identity/v3/services => generated 251 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:01.470367 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036fae9-77fa-4761-a90c-4839a8955171 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:01.471265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036fae9-77fa-4761-a90c-4839a8955171 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:01.471746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036fae9-77fa-4761-a90c-4839a8955171 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:01.474598 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 436/872] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:01.480440 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6b829-2789-49a4-81b6-11f185cf140e None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:01.480739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6b829-2789-49a4-81b6-11f185cf140e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:01.481013 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6b829-2789-49a4-81b6-11f185cf140e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:01.498552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a4f6b829-2789-49a4-81b6-11f185cf140e None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:01.499883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a4f6b829-2789-49a4-81b6-11f185cf140e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:01.550514 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8f226-f1b7-4521-97c4-fbb7a17adb3d None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:01.565537 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:01.594701 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 437/873] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:01 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:22:01.615338 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7514c9-aa44-407d-9497-011eaff51b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:01.616058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7514c9-aa44-407d-9497-011eaff51b21 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:01.616347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7514c9-aa44-407d-9497-011eaff51b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:01.616630 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7514c9-aa44-407d-9497-011eaff51b21 None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:01.617505 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7514c9-aa44-407d-9497-011eaff51b21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:01.617822 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7514c9-aa44-407d-9497-011eaff51b21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:01.619263 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7514c9-aa44-407d-9497-011eaff51b21 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:01.620249 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6b829-2789-49a4-81b6-11f185cf140e None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:01.633228 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 437/874] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:22:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:01.639487 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7514c9-aa44-407d-9497-011eaff51b21 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:01.730530 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:01.752833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6b829-2789-49a4-81b6-11f185cf140e None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:01.768077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7514c9-aa44-407d-9497-011eaff51b21 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:01.793990 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97882120-2542-4a2d-87f3-8b54653fb304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:01.795137 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97882120-2542-4a2d-87f3-8b54653fb304 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:01.795434 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97882120-2542-4a2d-87f3-8b54653fb304 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:01.795722 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97882120-2542-4a2d-87f3-8b54653fb304 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:01.797748 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97882120-2542-4a2d-87f3-8b54653fb304 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:01.799135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97882120-2542-4a2d-87f3-8b54653fb304 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:01.812030 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c040c-089b-4d23-807b-c8df110b0f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:01.812750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c040c-089b-4d23-807b-c8df110b0f3b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:01.813063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c040c-089b-4d23-807b-c8df110b0f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:01.813356 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c040c-089b-4d23-807b-c8df110b0f3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:01.814206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c040c-089b-4d23-807b-c8df110b0f3b None admin] RBAC: Adding query filter params (role.id=0e34a6f9b89a4e29951fb189abacb34a, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=cfc2f31f654e486d8505d80aad17d17c) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:01.814528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c040c-089b-4d23-807b-c8df110b0f3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:01.816033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 438/875] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 17:22:01 2022] POST /identity/v3/services => generated 269 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:22:01.816153 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c040c-089b-4d23-807b-c8df110b0f3b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:01.821200 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc23e1-bf5d-4182-b549-8936b6f1f0d0 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:01.821200 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc23e1-bf5d-4182-b549-8936b6f1f0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:01.821200 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc23e1-bf5d-4182-b549-8936b6f1f0d0 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:01.821200 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 439/876] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:01.825616 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644d6d2-1d5a-47cc-891b-630c322e1f11 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:01.826489 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644d6d2-1d5a-47cc-891b-630c322e1f11 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:01.826924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644d6d2-1d5a-47cc-891b-630c322e1f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:01.828975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 438/877] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:22:01 2022] GET /identity/v3/role_assignments?user.id=cfc2f31f654e486d8505d80aad17d17c&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=0e34a6f9b89a4e29951fb189abacb34a => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:01.857984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e644d6d2-1d5a-47cc-891b-630c322e1f11 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:01.860314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e644d6d2-1d5a-47cc-891b-630c322e1f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:01.922586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:01.969110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 440/878] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:01 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:22:01.992190 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c040c-089b-4d23-807b-c8df110b0f3b None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:02.116742 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c040c-089b-4d23-807b-c8df110b0f3b None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:02.153213 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd7406-672f-4d12-91b5-87aa9a8d5134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:02.154026 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd7406-672f-4d12-91b5-87aa9a8d5134 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:02.154314 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd7406-672f-4d12-91b5-87aa9a8d5134 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:02.154590 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd7406-672f-4d12-91b5-87aa9a8d5134 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:02.155273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd7406-672f-4d12-91b5-87aa9a8d5134 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:02.156656 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd7406-672f-4d12-91b5-87aa9a8d5134 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:02.159396 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:02.170204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-11fd7406-672f-4d12-91b5-87aa9a8d5134 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:22:02.170204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:02.170204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:22:02.170204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:22:02.170204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:22:02.170204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:22:02.170204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:22:02.170204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:02.170204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:22:02.170204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:02.170204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:22:02.170204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:02.170204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:02.170204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:02.170204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:02.170204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:02.170204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:02.172021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:02.172021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:02.172021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:02.172021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:02.172021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:02.172021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:02.172021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:22:02.172021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:22:02.172021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:22:02.172021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:22:02.172021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:02.172021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:02.172021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:02.172021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:02.172021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:02.172021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:02.173179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:02.173179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:02.173179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:02.173179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:02.173179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:02.173179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:02.173179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:02.173179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:02.173179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:02.173179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:02.173179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:22:02.173179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:02.173179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:22:02.173179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:02.176235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 439/879] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 17:22:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:02.181791 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644d6d2-1d5a-47cc-891b-630c322e1f11 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:02.295086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644d6d2-1d5a-47cc-891b-630c322e1f11 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:02.330866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc2041-4aab-4326-87cd-8eadf19a6e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:02.331551 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc2041-4aab-4326-87cd-8eadf19a6e4c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:02.331837 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc2041-4aab-4326-87cd-8eadf19a6e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:02.332119 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc2041-4aab-4326-87cd-8eadf19a6e4c None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:02.332836 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc2041-4aab-4326-87cd-8eadf19a6e4c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:02.333170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc2041-4aab-4326-87cd-8eadf19a6e4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:02.334555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc2041-4aab-4326-87cd-8eadf19a6e4c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:02.339822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 441/880] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jun 16 17:22:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:02.346546 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd7406-672f-4d12-91b5-87aa9a8d5134 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:02.462782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd7406-672f-4d12-91b5-87aa9a8d5134 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:02.503212 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fddb55c-a051-444b-aa59-b4fa33bf360b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:02.503885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fddb55c-a051-444b-aa59-b4fa33bf360b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:02.504170 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fddb55c-a051-444b-aa59-b4fa33bf360b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:02.504454 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fddb55c-a051-444b-aa59-b4fa33bf360b None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:02.505189 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fddb55c-a051-444b-aa59-b4fa33bf360b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:02.505508 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fddb55c-a051-444b-aa59-b4fa33bf360b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:02.506871 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fddb55c-a051-444b-aa59-b4fa33bf360b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:02.512188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 440/881] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jun 16 17:22:02 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:03.727363 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbcf7a-bdb9-4f1b-b882-1af4d5e1d076 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:03.727926 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbcf7a-bdb9-4f1b-b882-1af4d5e1d076 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:03.727926 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbcf7a-bdb9-4f1b-b882-1af4d5e1d076 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:03.729465 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 442/882] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:03.735694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cf86b-49fe-4d3e-ac15-f62873fd1e05 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:03.736599 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cf86b-49fe-4d3e-ac15-f62873fd1e05 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:03.737104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cf86b-49fe-4d3e-ac15-f62873fd1e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:03.759180 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-c91cf86b-49fe-4d3e-ac15-f62873fd1e05 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:03.761459 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-c91cf86b-49fe-4d3e-ac15-f62873fd1e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:03.835989 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:03.853259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 441/883] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:03.869928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbcf7a-bdb9-4f1b-b882-1af4d5e1d076 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:03.907459 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:03.982860 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbcf7a-bdb9-4f1b-b882-1af4d5e1d076 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:04.020696 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb0297-5e02-4a01-b489-aee4bcbd66d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:04.021543 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb0297-5e02-4a01-b489-aee4bcbd66d2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:04.021830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb0297-5e02-4a01-b489-aee4bcbd66d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:04.022110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb0297-5e02-4a01-b489-aee4bcbd66d2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:04.022773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb0297-5e02-4a01-b489-aee4bcbd66d2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:04.024140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb0297-5e02-4a01-b489-aee4bcbd66d2 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:04.036987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-10eb0297-5e02-4a01-b489-aee4bcbd66d2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:22:04.036987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:04.036987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:22:04.036987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:22:04.036987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:22:04.036987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:22:04.036987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:22:04.036987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:04.036987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:22:04.036987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:04.036987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:22:04.036987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:04.036987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:04.036987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:04.036987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:04.036987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:04.036987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:04.038087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:04.038087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:04.038087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:04.038087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:04.038087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:04.038087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:04.038087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:22:04.038087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:22:04.038087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:22:04.038087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:22:04.038087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:04.038087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:04.038087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:04.038087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:04.038087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:04.038087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:04.038953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:04.038953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:04.038953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:04.038953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:04.038953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:04.038953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:04.038953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:04.038953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:04.038953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:04.038953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:04.038953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:22:04.038953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:04.038953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:22:04.038953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:04.038953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 443/884] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:22:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:04.044433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91cf86b-49fe-4d3e-ac15-f62873fd1e05 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:04.159338 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91cf86b-49fe-4d3e-ac15-f62873fd1e05 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:04.178339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c1a4f-d651-416d-9b7a-77d48cd2efd3 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:04.178677 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c1a4f-d651-416d-9b7a-77d48cd2efd3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:04.178905 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c1a4f-d651-416d-9b7a-77d48cd2efd3 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:04.180537 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 444/885] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:04.185247 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d7a9d-4aab-4482-968d-bed582297334 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:04.185538 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d7a9d-4aab-4482-968d-bed582297334 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:04.185814 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d7a9d-4aab-4482-968d-bed582297334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:04.197441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef405b97-77f3-425b-87da-5f7691a70f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:04.198527 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef405b97-77f3-425b-87da-5f7691a70f60 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:04.199023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef405b97-77f3-425b-87da-5f7691a70f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:04.199514 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef405b97-77f3-425b-87da-5f7691a70f60 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:04.200486 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef405b97-77f3-425b-87da-5f7691a70f60 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:04.201026 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef405b97-77f3-425b-87da-5f7691a70f60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:04.203480 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef405b97-77f3-425b-87da-5f7691a70f60 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:04.209909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 442/886] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:22:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:04.218555 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef405b97-77f3-425b-87da-5f7691a70f60 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:04.300080 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:04.338933 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-249d7a9d-4aab-4482-968d-bed582297334 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:04.340205 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-249d7a9d-4aab-4482-968d-bed582297334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:04.346588 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef405b97-77f3-425b-87da-5f7691a70f60 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:04.380921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8eb37-b370-452e-88f3-c24f0012a4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:04.381913 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8eb37-b370-452e-88f3-c24f0012a4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:04.382199 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8eb37-b370-452e-88f3-c24f0012a4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:04.382477 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8eb37-b370-452e-88f3-c24f0012a4c0 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:04.383464 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8eb37-b370-452e-88f3-c24f0012a4c0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:04.383775 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8eb37-b370-452e-88f3-c24f0012a4c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:04.385136 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8eb37-b370-452e-88f3-c24f0012a4c0 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:04.395354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 443/887] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:22:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:04.395354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4afbc-aa20-47f5-beb6-f393c056281b None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:04.395354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4afbc-aa20-47f5-beb6-f393c056281b None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:04.395354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4afbc-aa20-47f5-beb6-f393c056281b None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:04.395354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 444/888] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:04.398395 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4afbc-aa20-47f5-beb6-f393c056281b None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:04.422684 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 445/889] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:22:04.426483 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4bbe4-de5d-495a-8b18-f9bac2c3c8ab None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:04.426774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4bbe4-de5d-495a-8b18-f9bac2c3c8ab None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:04.427074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4bbe4-de5d-495a-8b18-f9bac2c3c8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:04.449254 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-24b4bbe4-de5d-495a-8b18-f9bac2c3c8ab None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:04.451091 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-24b4bbe4-de5d-495a-8b18-f9bac2c3c8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:04.473753 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:04.529275 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 446/890] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:04.529275 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4bbe4-de5d-495a-8b18-f9bac2c3c8ab None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:04.537856 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4afbc-aa20-47f5-beb6-f393c056281b None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:04.563403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:04.580137 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d6253-8e88-4978-b852-dc54c784aa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:04.580952 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d6253-8e88-4978-b852-dc54c784aa69 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:04.581405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d6253-8e88-4978-b852-dc54c784aa69 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:04.581690 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d6253-8e88-4978-b852-dc54c784aa69 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:04.582760 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d6253-8e88-4978-b852-dc54c784aa69 None admin] RBAC: Adding query filter params (interface=public, service_id=072974aa58eb4bd5a5ab16b6c35e24d4, region_id=RegionOne) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:04.583439 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d6253-8e88-4978-b852-dc54c784aa69 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:04.584833 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d6253-8e88-4978-b852-dc54c784aa69 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:04.593493 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 445/891] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jun 16 17:22:04 2022] GET /identity/v3/endpoints?service_id=072974aa58eb4bd5a5ab16b6c35e24d4&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:04.594472 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d6253-8e88-4978-b852-dc54c784aa69 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:04.643920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4bbe4-de5d-495a-8b18-f9bac2c3c8ab None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:04.684219 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41698544-c54b-4e49-9e12-c19ba7a2ba54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:04.685513 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41698544-c54b-4e49-9e12-c19ba7a2ba54 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:04.686024 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41698544-c54b-4e49-9e12-c19ba7a2ba54 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:04.687030 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41698544-c54b-4e49-9e12-c19ba7a2ba54 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:04.688413 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41698544-c54b-4e49-9e12-c19ba7a2ba54 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:04.690193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41698544-c54b-4e49-9e12-c19ba7a2ba54 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:04.704683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-41698544-c54b-4e49-9e12-c19ba7a2ba54 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:22:04.704683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:04.704683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:22:04.704683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:22:04.704683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:22:04.704683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:22:04.704683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:22:04.704683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:04.704683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:22:04.704683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:04.704683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:22:04.704683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:04.704683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:04.704683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:04.704683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:04.704683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:04.705775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:04.705775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:04.705775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:04.705775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:04.705775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:04.705775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:04.705775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:04.705775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:22:04.705775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:22:04.705775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:22:04.705775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:22:04.705775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:04.705775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:04.705775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:04.705775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:04.705775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:04.706649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:04.706649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:04.706649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:04.706649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:04.706649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:04.706649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:04.706649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:04.706649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:04.706649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:04.706649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:04.706649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:04.706649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:22:04.706649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:04.706649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:22:04.706649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:04.707467 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 447/892] 10.222.0.40 () {56 vars in 1132 bytes} [Thu Jun 16 17:22:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:04.707543 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41698544-c54b-4e49-9e12-c19ba7a2ba54 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:04.714311 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d6253-8e88-4978-b852-dc54c784aa69 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:04.732018 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:04.751168 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea03e3-f46a-484f-952f-97418e289661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:04.751985 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea03e3-f46a-484f-952f-97418e289661 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:04.752272 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea03e3-f46a-484f-952f-97418e289661 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:04.752551 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea03e3-f46a-484f-952f-97418e289661 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:04.762777 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea03e3-f46a-484f-952f-97418e289661 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:22:04.771545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-1dea03e3-f46a-484f-952f-97418e289661 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:22:04.771545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:04.771545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:04.771545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:04.771545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:04.771545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:04.771545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:04.771545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:04.771545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:04.771545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:04.771545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:04.771545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:04.771545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:22:04.771545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:22:04.771545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:22:04.771545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:22:04.772528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:22:04.772528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:22:04.772528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:22:04.772528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:22:04.772528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:04.772528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:04.772528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:04.772528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:04.772528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:04.772528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:04.772528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:04.772528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:04.772528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:04.772528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:04.772528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:04.772528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:04.773445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:04.773445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:04.773445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:04.773445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:04.773445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:22:04.773445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:22:04.773445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:22:04.773445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:22:04.773445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:22:04.773445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:22:04.773445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:22:04.773445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:04.773445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 446/893] 10.222.0.40 () {56 vars in 1120 bytes} [Thu Jun 16 17:22:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:04.774156 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea03e3-f46a-484f-952f-97418e289661 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:04.817131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41698544-c54b-4e49-9e12-c19ba7a2ba54 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:04.822021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:04.851330 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d94f9d-1e9c-430d-ad42-73adee29f1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:04.852005 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d94f9d-1e9c-430d-ad42-73adee29f1de None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:04.852290 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d94f9d-1e9c-430d-ad42-73adee29f1de None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:04.852569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d94f9d-1e9c-430d-ad42-73adee29f1de None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:04.853288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d94f9d-1e9c-430d-ad42-73adee29f1de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:04.854648 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d94f9d-1e9c-430d-ad42-73adee29f1de None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:04.860443 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 448/894] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jun 16 17:22:04 2022] GET /identity/v3/services => generated 1268 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:04.861679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c010a0-ae87-4d66-866f-810147036f82 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:04.861970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c010a0-ae87-4d66-866f-810147036f82 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:04.862298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c010a0-ae87-4d66-866f-810147036f82 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:04.865205 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 449/895] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:04.865205 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c010a0-ae87-4d66-866f-810147036f82 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:04.884160 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea03e3-f46a-484f-952f-97418e289661 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:04.927595 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355bc0d-a16e-4b19-bac8-63fd2c269b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:04.928290 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355bc0d-a16e-4b19-bac8-63fd2c269b7b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:04.928583 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355bc0d-a16e-4b19-bac8-63fd2c269b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:04.928867 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355bc0d-a16e-4b19-bac8-63fd2c269b7b None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:04.929632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355bc0d-a16e-4b19-bac8-63fd2c269b7b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:04.929952 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355bc0d-a16e-4b19-bac8-63fd2c269b7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:04.931370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355bc0d-a16e-4b19-bac8-63fd2c269b7b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:04.944677 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 447/896] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 17:22:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:04.944677 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371b581-9e50-40e7-abfc-c57f9b3145da None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:04.944677 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371b581-9e50-40e7-abfc-c57f9b3145da None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:04.944677 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371b581-9e50-40e7-abfc-c57f9b3145da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:04.962378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-9371b581-9e50-40e7-abfc-c57f9b3145da None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:04.963709 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-9371b581-9e50-40e7-abfc-c57f9b3145da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:04.982777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:05.003733 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c010a0-ae87-4d66-866f-810147036f82 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:05.042682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 448/897] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:05.044732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371b581-9e50-40e7-abfc-c57f9b3145da None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:05.048535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978e852-75f5-4259-a935-479eb97584ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:05.049259 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978e852-75f5-4259-a935-479eb97584ae None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:05.049547 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978e852-75f5-4259-a935-479eb97584ae None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:05.049822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978e852-75f5-4259-a935-479eb97584ae None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:05.050578 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978e852-75f5-4259-a935-479eb97584ae None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:05.050877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978e852-75f5-4259-a935-479eb97584ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:05.053442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978e852-75f5-4259-a935-479eb97584ae None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:05.065308 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 450/898] 10.222.0.40 () {56 vars in 1115 bytes} [Thu Jun 16 17:22:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:05.069125 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978e852-75f5-4259-a935-479eb97584ae None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:05.102983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:05.174545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371b581-9e50-40e7-abfc-c57f9b3145da None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:05.190389 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978e852-75f5-4259-a935-479eb97584ae None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:05.211375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eeedb9-5036-42fe-8e17-2961ccf8e3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:05.212052 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eeedb9-5036-42fe-8e17-2961ccf8e3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:05.212339 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eeedb9-5036-42fe-8e17-2961ccf8e3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:05.212619 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eeedb9-5036-42fe-8e17-2961ccf8e3c7 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:05.213372 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eeedb9-5036-42fe-8e17-2961ccf8e3c7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:05.213687 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eeedb9-5036-42fe-8e17-2961ccf8e3c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:05.215086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eeedb9-5036-42fe-8e17-2961ccf8e3c7 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:05.220849 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 449/899] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 17:22:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:05.221659 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eeedb9-5036-42fe-8e17-2961ccf8e3c7 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:05.240319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53df43-8fe7-48b2-9f1f-dbd1a3754c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:05.241163 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53df43-8fe7-48b2-9f1f-dbd1a3754c7e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:05.241470 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53df43-8fe7-48b2-9f1f-dbd1a3754c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:05.241753 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53df43-8fe7-48b2-9f1f-dbd1a3754c7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:05.253555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53df43-8fe7-48b2-9f1f-dbd1a3754c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:05.255013 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53df43-8fe7-48b2-9f1f-dbd1a3754c7e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:05.263671 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:05.279256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-4c53df43-8fe7-48b2-9f1f-dbd1a3754c7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:05.279256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:05.279256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:05.279256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:05.279256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:05.279256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:05.279256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:05.279256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:05.279256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:05.279256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:05.279256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:05.279256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:05.279256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:22:05.279256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:22:05.279256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:22:05.280783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:22:05.280783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:05.280783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:05.280783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:05.280783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:05.280783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:05.280783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:05.280783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:05.280783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:05.280783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:05.280783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:05.280783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:05.280783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:05.280783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:05.280783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:05.280783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:05.280783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:05.287067 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:22:05.287067 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:22:05.287067 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:05.287067 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:05.287067 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 451/900] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:22:05.287067 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53df43-8fe7-48b2-9f1f-dbd1a3754c7e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:05.338764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eeedb9-5036-42fe-8e17-2961ccf8e3c7 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:05.341947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:05.378722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788820eb-7d1c-4f51-833e-29883cff26a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:05.379286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788820eb-7d1c-4f51-833e-29883cff26a7 None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:05.379582 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788820eb-7d1c-4f51-833e-29883cff26a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:05.379958 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788820eb-7d1c-4f51-833e-29883cff26a7 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:05.380598 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788820eb-7d1c-4f51-833e-29883cff26a7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:05.382016 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788820eb-7d1c-4f51-833e-29883cff26a7 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:05.397435 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 450/901] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 17:22:05 2022] POST /identity/v3/services => generated 241 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:05.398164 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788820eb-7d1c-4f51-833e-29883cff26a7 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:05.407999 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53df43-8fe7-48b2-9f1f-dbd1a3754c7e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:05.466611 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a4d808-551c-4784-b05c-fcf2890c6ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:05.469333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4d808-551c-4784-b05c-fcf2890c6ecf None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:05.469863 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4d808-551c-4784-b05c-fcf2890c6ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:05.470407 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4d808-551c-4784-b05c-fcf2890c6ecf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:05.472031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a4d808-551c-4784-b05c-fcf2890c6ecf None admin] RBAC: Adding query filter params (interface=public, service_id=ee8e4f0a3e4d4729bd1956149d9f7636, region_id=RegionOne) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:05.473048 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a4d808-551c-4784-b05c-fcf2890c6ecf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:05.483095 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a4d808-551c-4784-b05c-fcf2890c6ecf None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:05.497060 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 452/902] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jun 16 17:22:05 2022] GET /identity/v3/endpoints?service_id=ee8e4f0a3e4d4729bd1956149d9f7636&interface=public®ion_id=RegionOne => generated 194 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:05.498908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a4d808-551c-4784-b05c-fcf2890c6ecf None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:05.549118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788820eb-7d1c-4f51-833e-29883cff26a7 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:05.590061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:05.593350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7386021-6279-4836-89c0-b0e3789a5c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:05.593758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7386021-6279-4836-89c0-b0e3789a5c93 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:05.594066 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7386021-6279-4836-89c0-b0e3789a5c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:05.594328 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7386021-6279-4836-89c0-b0e3789a5c93 None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:05.595626 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7386021-6279-4836-89c0-b0e3789a5c93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:05.596293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7386021-6279-4836-89c0-b0e3789a5c93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:05.597654 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7386021-6279-4836-89c0-b0e3789a5c93 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:05.614346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 451/903] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:05.619645 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7386021-6279-4836-89c0-b0e3789a5c93 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:05.646439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a4d808-551c-4784-b05c-fcf2890c6ecf None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:05.650301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:05.695764 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7509e-bc18-478c-ab15-60644cd90d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:05.696551 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7509e-bc18-478c-ab15-60644cd90d27 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:05.696914 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7509e-bc18-478c-ab15-60644cd90d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:05.697299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7509e-bc18-478c-ab15-60644cd90d27 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:05.698071 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7509e-bc18-478c-ab15-60644cd90d27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:05.699576 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7509e-bc18-478c-ab15-60644cd90d27 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:05.713297 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 453/904] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jun 16 17:22:05 2022] GET /identity/v3/services => generated 1497 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:22:05.750614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7386021-6279-4836-89c0-b0e3789a5c93 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:05.792330 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269456ef-a243-4e4c-a2ec-9a25686a893c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:05.793188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269456ef-a243-4e4c-a2ec-9a25686a893c None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:05.793473 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269456ef-a243-4e4c-a2ec-9a25686a893c None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:05.793750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269456ef-a243-4e4c-a2ec-9a25686a893c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:05.802572 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269456ef-a243-4e4c-a2ec-9a25686a893c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:05.804105 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269456ef-a243-4e4c-a2ec-9a25686a893c None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:05.821022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-269456ef-a243-4e4c-a2ec-9a25686a893c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:22:05.821022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:05.821022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:05.821022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:05.821022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:05.821022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:05.821022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:05.821022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:05.821022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:05.821022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:05.821022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:05.821022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:05.821022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:22:05.821022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:22:05.821022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:22:05.821022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:22:05.822133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:05.822133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:05.822133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:22:05.822133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:22:05.822133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:22:05.822133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:22:05.822133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:05.822133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:05.822133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:05.822133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:05.822133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:05.822133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:05.822133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:05.822133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:05.822133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:05.822133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:05.822133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:05.822133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:05.823172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:05.823172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:05.823172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:22:05.823172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:22:05.823172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:22:05.823172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:22:05.823172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:22:05.823172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:22:05.823172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:22:05.823172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:05.823172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 452/905] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jun 16 17:22:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:22:05.831436 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7509e-bc18-478c-ab15-60644cd90d27 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:05.942079 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7509e-bc18-478c-ab15-60644cd90d27 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:05.965784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:05.978267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836db2e2-ce3e-44c4-8b53-1138dd209554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:05.980820 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836db2e2-ce3e-44c4-8b53-1138dd209554 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:05.981107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836db2e2-ce3e-44c4-8b53-1138dd209554 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:05.981421 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836db2e2-ce3e-44c4-8b53-1138dd209554 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:05.982309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836db2e2-ce3e-44c4-8b53-1138dd209554 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:05.982623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836db2e2-ce3e-44c4-8b53-1138dd209554 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:05.984316 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836db2e2-ce3e-44c4-8b53-1138dd209554 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:06.005126 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 454/906] 10.222.0.40 () {56 vars in 1149 bytes} [Thu Jun 16 17:22:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:06.010991 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269456ef-a243-4e4c-a2ec-9a25686a893c None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:06.105092 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:06.125537 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269456ef-a243-4e4c-a2ec-9a25686a893c None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:06.164836 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8284a7d-abe9-49fe-8944-bbc1236450c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:06.165522 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8284a7d-abe9-49fe-8944-bbc1236450c8 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:06.165816 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8284a7d-abe9-49fe-8944-bbc1236450c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:06.166101 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8284a7d-abe9-49fe-8944-bbc1236450c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:06.240747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8284a7d-abe9-49fe-8944-bbc1236450c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:06.242340 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8284a7d-abe9-49fe-8944-bbc1236450c8 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:06.334681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:06.353438 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 453/907] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:22:06 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:22:06.358974 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836db2e2-ce3e-44c4-8b53-1138dd209554 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:06.461643 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836db2e2-ce3e-44c4-8b53-1138dd209554 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:06.497104 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddec2a2-5581-4759-9970-18208148b026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:06.497854 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddec2a2-5581-4759-9970-18208148b026 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:06.498139 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddec2a2-5581-4759-9970-18208148b026 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:06.498419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddec2a2-5581-4759-9970-18208148b026 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:06.509923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddec2a2-5581-4759-9970-18208148b026 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:06.511367 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddec2a2-5581-4759-9970-18208148b026 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:06.536945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-7ddec2a2-5581-4759-9970-18208148b026 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:06.536945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:06.536945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:06.536945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:06.536945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:06.536945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:06.536945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:06.536945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:06.536945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:06.536945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:06.536945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:06.536945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:06.536945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:22:06.536945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:22:06.536945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:22:06.538550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:22:06.538550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:06.538550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:06.538550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:06.538550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:06.538550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:06.538550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:06.538550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:06.538550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:06.538550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:06.538550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:06.538550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:06.538550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:06.538550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:06.538550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:06.538550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:06.538550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:06.540094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:22:06.540094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:22:06.540094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:06.540094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:06.540094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 455/908] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:22:06.544138 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8284a7d-abe9-49fe-8944-bbc1236450c8 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:06.658692 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8284a7d-abe9-49fe-8944-bbc1236450c8 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:06.697621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef69eb2-9023-4a6f-bb37-2a4b9e1952ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:06.698298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef69eb2-9023-4a6f-bb37-2a4b9e1952ad None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:06.698588 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef69eb2-9023-4a6f-bb37-2a4b9e1952ad None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:06.698873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef69eb2-9023-4a6f-bb37-2a4b9e1952ad None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:06.699622 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef69eb2-9023-4a6f-bb37-2a4b9e1952ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:06.699938 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef69eb2-9023-4a6f-bb37-2a4b9e1952ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:06.701344 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef69eb2-9023-4a6f-bb37-2a4b9e1952ad None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:06.719326 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 454/909] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:06.720701 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddec2a2-5581-4759-9970-18208148b026 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:06.755569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:06.832403 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddec2a2-5581-4759-9970-18208148b026 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:06.870981 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84444361-01dd-4388-9ca2-37dc9a2fc350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:06.871790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84444361-01dd-4388-9ca2-37dc9a2fc350 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:06.872080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84444361-01dd-4388-9ca2-37dc9a2fc350 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:06.872361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84444361-01dd-4388-9ca2-37dc9a2fc350 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:06.879938 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84444361-01dd-4388-9ca2-37dc9a2fc350 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:06.881429 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84444361-01dd-4388-9ca2-37dc9a2fc350 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:06.896526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-84444361-01dd-4388-9ca2-37dc9a2fc350 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:22:06.896526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:06.896526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:06.896526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:06.896526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:06.896526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:06.896526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:06.896526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:06.896526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:06.896526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:06.896526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:06.896526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:06.896526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:22:06.896526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:22:06.896526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:22:06.898654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:22:06.898654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:06.898654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:06.898654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:06.898654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:06.898654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:06.898654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:06.898654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:06.898654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:06.898654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:06.898654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:06.898654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:06.898654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:06.898654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:06.898654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:06.898654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:06.898654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:06.900830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:22:06.900830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:22:06.900830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:22:06.900830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:22:06.900830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:22:06.900830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:22:06.900830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:22:06.900830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:06.905505 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 456/910] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:22:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:06.905735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef69eb2-9023-4a6f-bb37-2a4b9e1952ad None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:06.958052 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:07.024420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef69eb2-9023-4a6f-bb37-2a4b9e1952ad None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:07.067306 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2db67b-7bc7-424d-a2ed-c80a40543936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:07.067974 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2db67b-7bc7-424d-a2ed-c80a40543936 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:07.068257 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2db67b-7bc7-424d-a2ed-c80a40543936 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:07.068525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2db67b-7bc7-424d-a2ed-c80a40543936 None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:07.069350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2db67b-7bc7-424d-a2ed-c80a40543936 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:07.069665 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2db67b-7bc7-424d-a2ed-c80a40543936 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:07.071094 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2db67b-7bc7-424d-a2ed-c80a40543936 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:07.073018 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ee546-af93-4a07-a3f6-0f4e5b2eb253 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:07.073371 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ee546-af93-4a07-a3f6-0f4e5b2eb253 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:07.073659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ee546-af93-4a07-a3f6-0f4e5b2eb253 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:07.075266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 457/911] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:07.080514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd68b6-7b8a-45a8-9ba3-ea7ef7dccc56 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:07.080805 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd68b6-7b8a-45a8-9ba3-ea7ef7dccc56 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:07.081094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd68b6-7b8a-45a8-9ba3-ea7ef7dccc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:07.084062 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 455/912] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:22:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:07.089770 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2db67b-7bc7-424d-a2ed-c80a40543936 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:07.101259 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-68fd68b6-7b8a-45a8-9ba3-ea7ef7dccc56 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:07.102466 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-68fd68b6-7b8a-45a8-9ba3-ea7ef7dccc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:07.181250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 458/913] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:07.198069 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd68b6-7b8a-45a8-9ba3-ea7ef7dccc56 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:07.209832 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2db67b-7bc7-424d-a2ed-c80a40543936 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:07.225766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:07.238823 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:07.261279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14f876-a209-4606-9d0e-b65d55b56086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:07.262619 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14f876-a209-4606-9d0e-b65d55b56086 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:07.263093 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14f876-a209-4606-9d0e-b65d55b56086 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:07.263578 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14f876-a209-4606-9d0e-b65d55b56086 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:07.265233 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14f876-a209-4606-9d0e-b65d55b56086 None admin] RBAC: Adding query filter params (role.id=3a41c48a11174f87a500cc1d6618ee65, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=cfc2f31f654e486d8505d80aad17d17c) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:07.265792 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14f876-a209-4606-9d0e-b65d55b56086 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:07.268436 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14f876-a209-4606-9d0e-b65d55b56086 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:07.277376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 456/914] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:22:07 2022] GET /identity/v3/role_assignments?user.id=cfc2f31f654e486d8505d80aad17d17c&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=3a41c48a11174f87a500cc1d6618ee65 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:07.343272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd68b6-7b8a-45a8-9ba3-ea7ef7dccc56 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:07.384615 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df863664-df10-4574-9d39-c7e5209a3fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:07.385471 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df863664-df10-4574-9d39-c7e5209a3fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:07.385761 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df863664-df10-4574-9d39-c7e5209a3fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:07.386044 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df863664-df10-4574-9d39-c7e5209a3fd5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:07.386728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df863664-df10-4574-9d39-c7e5209a3fd5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:07.388128 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df863664-df10-4574-9d39-c7e5209a3fd5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:07.401720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-df863664-df10-4574-9d39-c7e5209a3fd5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:22:07.401720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:07.401720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:22:07.401720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:22:07.401720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:22:07.401720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:22:07.401720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:22:07.401720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:07.401720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:22:07.401720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:07.401720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:22:07.401720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:07.401720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:07.401720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:07.401720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:07.401720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:07.401720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:07.402819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:07.402819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:07.402819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:07.402819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:07.402819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:07.402819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:07.402819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:22:07.402819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:22:07.402819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:22:07.402819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:22:07.402819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:07.402819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:07.402819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:07.402819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:07.402819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:07.402819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:07.403683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:07.403683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:07.403683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:07.403683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:07.403683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:07.403683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:07.403683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:07.403683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:07.403683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:07.403683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:07.403683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:22:07.403683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:07.403683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:22:07.403683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:07.403683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 459/915] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:22:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:07.411726 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14f876-a209-4606-9d0e-b65d55b56086 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:07.528012 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14f876-a209-4606-9d0e-b65d55b56086 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:07.564173 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f0c6b-db7e-483c-8b8f-5801823e67d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:07.565261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f0c6b-db7e-483c-8b8f-5801823e67d6 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:07.565771 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f0c6b-db7e-483c-8b8f-5801823e67d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:07.566251 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f0c6b-db7e-483c-8b8f-5801823e67d6 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:07.567169 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f0c6b-db7e-483c-8b8f-5801823e67d6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:07.567678 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f0c6b-db7e-483c-8b8f-5801823e67d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:07.569269 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f0c6b-db7e-483c-8b8f-5801823e67d6 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:07.575023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 457/916] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:22:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:07.581471 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df863664-df10-4574-9d39-c7e5209a3fd5 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:07.673611 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:07.691259 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df863664-df10-4574-9d39-c7e5209a3fd5 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:07.731563 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86005de5-97f9-4fb6-a10a-daf6799ce0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:07.732623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86005de5-97f9-4fb6-a10a-daf6799ce0ad None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:07.733167 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86005de5-97f9-4fb6-a10a-daf6799ce0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:07.733680 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86005de5-97f9-4fb6-a10a-daf6799ce0ad None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:07.734687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86005de5-97f9-4fb6-a10a-daf6799ce0ad None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:07.735197 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86005de5-97f9-4fb6-a10a-daf6799ce0ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:07.736786 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86005de5-97f9-4fb6-a10a-daf6799ce0ad None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:07.742633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 460/917] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 17:22:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:07.748553 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f0c6b-db7e-483c-8b8f-5801823e67d6 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:07.765180 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55efe7f-ad6f-468b-ab0c-316c86df1b42 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:07.765509 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55efe7f-ad6f-468b-ab0c-316c86df1b42 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:07.765756 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55efe7f-ad6f-468b-ab0c-316c86df1b42 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:07.767709 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 461/918] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:22:07.773663 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fed838-5c45-4641-96e8-8c3fd9fd7f9b None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:07.774163 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fed838-5c45-4641-96e8-8c3fd9fd7f9b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:07.774628 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fed838-5c45-4641-96e8-8c3fd9fd7f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:07.787135 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:07.795719 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-78fed838-5c45-4641-96e8-8c3fd9fd7f9b None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:07.797394 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-78fed838-5c45-4641-96e8-8c3fd9fd7f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:07.876941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f0c6b-db7e-483c-8b8f-5801823e67d6 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:07.889122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 462/919] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:22:07.912932 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fed838-5c45-4641-96e8-8c3fd9fd7f9b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:07.929401 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1fcffe-8563-4361-b1ff-91a77deb1833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:07.930888 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fcffe-8563-4361-b1ff-91a77deb1833 None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:07.931398 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fcffe-8563-4361-b1ff-91a77deb1833 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:07.931903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fcffe-8563-4361-b1ff-91a77deb1833 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:07.932887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1fcffe-8563-4361-b1ff-91a77deb1833 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:07.934632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1fcffe-8563-4361-b1ff-91a77deb1833 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:07.969700 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 458/920] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jun 16 17:22:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:08.023826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:08.052045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fed838-5c45-4641-96e8-8c3fd9fd7f9b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:08.100582 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bcdac4-cdab-4733-8d65-a939c119944e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:08.101431 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcdac4-cdab-4733-8d65-a939c119944e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:08.101713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcdac4-cdab-4733-8d65-a939c119944e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:08.101988 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcdac4-cdab-4733-8d65-a939c119944e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:08.102653 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bcdac4-cdab-4733-8d65-a939c119944e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:08.103637 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe0816-dea7-40bd-a9a6-9691ce748e7c None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:08.103934 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe0816-dea7-40bd-a9a6-9691ce748e7c None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:08.104109 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bcdac4-cdab-4733-8d65-a939c119944e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:08.104215 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe0816-dea7-40bd-a9a6-9691ce748e7c None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:08.105916 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 459/921] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:08.112382 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5057b-9730-499f-9120-fdcc8146d3fc None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:08.112700 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5057b-9730-499f-9120-fdcc8146d3fc None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:08.112963 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5057b-9730-499f-9120-fdcc8146d3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:08.117652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-b6bcdac4-cdab-4733-8d65-a939c119944e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:22:08.117652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:08.117652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:22:08.117652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:22:08.117652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:22:08.117652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:22:08.117652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:22:08.117652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:08.117652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:22:08.117652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:08.117652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:22:08.117652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:08.117652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:08.117652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:08.117652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:08.117652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:08.117652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:08.118686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:08.118686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:08.118686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:08.118686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:08.118686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:08.118686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:08.118686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:22:08.118686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:22:08.118686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:22:08.118686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:22:08.118686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:08.118686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:08.118686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:08.118686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:08.118686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:08.118686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:08.119586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:08.119586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:08.119586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:08.119586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:08.119586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:08.119586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:08.119586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:08.119586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:08.119586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:08.119586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:08.119586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:22:08.119586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:08.119586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:22:08.119586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:08.119586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 463/922] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 17:22:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:22:08.127044 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bcdac4-cdab-4733-8d65-a939c119944e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:08.140203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-1be5057b-9730-499f-9120-fdcc8146d3fc None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:08.142086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-1be5057b-9730-499f-9120-fdcc8146d3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:08.242882 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 460/923] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:22:08.258010 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bcdac4-cdab-4733-8d65-a939c119944e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:08.268349 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5057b-9730-499f-9120-fdcc8146d3fc None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:08.290391 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:08.302995 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0768a0b-4a51-475d-92a9-2e03fe767fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:08.303761 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0768a0b-4a51-475d-92a9-2e03fe767fae None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:08.304054 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0768a0b-4a51-475d-92a9-2e03fe767fae None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:08.304337 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0768a0b-4a51-475d-92a9-2e03fe767fae None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:08.305059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0768a0b-4a51-475d-92a9-2e03fe767fae None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:08.305499 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0768a0b-4a51-475d-92a9-2e03fe767fae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:08.307000 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0768a0b-4a51-475d-92a9-2e03fe767fae None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:08.313622 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 464/924] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jun 16 17:22:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:08.319028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0768a0b-4a51-475d-92a9-2e03fe767fae None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:08.340723 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:08.381923 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5057b-9730-499f-9120-fdcc8146d3fc None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:08.418866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fdcc0-6199-492a-be60-c712b986a868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:08.419696 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fdcc0-6199-492a-be60-c712b986a868 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:08.419985 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fdcc0-6199-492a-be60-c712b986a868 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:08.420265 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fdcc0-6199-492a-be60-c712b986a868 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:08.421009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5fdcc0-6199-492a-be60-c712b986a868 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:08.422440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5fdcc0-6199-492a-be60-c712b986a868 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:08.435172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-4e5fdcc0-6199-492a-be60-c712b986a868 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:22:08.435172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:08.435172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:22:08.435172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:22:08.435172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:22:08.435172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:22:08.435172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:22:08.435172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:08.435172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:22:08.435172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:08.435172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:22:08.435172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:08.435172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:08.435172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:08.435172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:08.435172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:08.436949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:08.436949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:08.436949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:08.436949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:08.436949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:08.436949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:08.436949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:08.436949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:22:08.436949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:22:08.436949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:22:08.436949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:22:08.436949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:08.436949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:08.436949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:08.436949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:08.436949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:08.438273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:08.438273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:08.438273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:08.438273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:08.438273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:08.438273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:08.438273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:08.438273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:08.438273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:08.438273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:08.438273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:08.438273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:22:08.438273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:08.438273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:22:08.438273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:08.441414 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 461/925] 10.222.0.40 () {56 vars in 1132 bytes} [Thu Jun 16 17:22:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:08.442990 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fdcc0-6199-492a-be60-c712b986a868 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:08.460200 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0768a0b-4a51-475d-92a9-2e03fe767fae None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:08.510783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f6ad5-a168-441c-a7fd-1118268b776a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:08.512126 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f6ad5-a168-441c-a7fd-1118268b776a None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:08.512568 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f6ad5-a168-441c-a7fd-1118268b776a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:08.513001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f6ad5-a168-441c-a7fd-1118268b776a None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:08.514133 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f6ad5-a168-441c-a7fd-1118268b776a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:08.515236 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f6ad5-a168-441c-a7fd-1118268b776a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:08.517915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f6ad5-a168-441c-a7fd-1118268b776a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:08.524341 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 465/926] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jun 16 17:22:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:08.530605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f6ad5-a168-441c-a7fd-1118268b776a None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:08.535734 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:08.560329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fdcc0-6199-492a-be60-c712b986a868 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:08.608913 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9d7b4-c7f0-4299-afa0-911f1180da46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:08.609698 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9d7b4-c7f0-4299-afa0-911f1180da46 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:08.610303 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9d7b4-c7f0-4299-afa0-911f1180da46 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:08.610598 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9d7b4-c7f0-4299-afa0-911f1180da46 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:08.611339 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b9d7b4-c7f0-4299-afa0-911f1180da46 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:08.611928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b9d7b4-c7f0-4299-afa0-911f1180da46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:08.614229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b9d7b4-c7f0-4299-afa0-911f1180da46 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:08.620683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:08.622617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 462/927] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 17:22:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:08.628872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9d7b4-c7f0-4299-afa0-911f1180da46 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:08.676306 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f6ad5-a168-441c-a7fd-1118268b776a None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:08.722336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2ef18-15ee-48f1-ae4c-c69de68537b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:08.723462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2ef18-15ee-48f1-ae4c-c69de68537b2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:08.723757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2ef18-15ee-48f1-ae4c-c69de68537b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:08.724069 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2ef18-15ee-48f1-ae4c-c69de68537b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:08.725563 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2ef18-15ee-48f1-ae4c-c69de68537b2 None admin] RBAC: Adding query filter params (interface=public, service_id=a1d7529d66a44838989ce4e5b6f3c307, region_id=RegionOne) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:08.725892 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2ef18-15ee-48f1-ae4c-c69de68537b2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:08.727536 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2ef18-15ee-48f1-ae4c-c69de68537b2 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:08.737241 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 466/928] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jun 16 17:22:08 2022] GET /identity/v3/endpoints?service_id=a1d7529d66a44838989ce4e5b6f3c307&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:08.741476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2ef18-15ee-48f1-ae4c-c69de68537b2 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:08.755592 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9d7b4-c7f0-4299-afa0-911f1180da46 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:08.804790 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911435f-fda0-481b-9492-6747a3045406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:08.805950 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911435f-fda0-481b-9492-6747a3045406 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:08.806248 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911435f-fda0-481b-9492-6747a3045406 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:08.806540 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911435f-fda0-481b-9492-6747a3045406 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:08.807537 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0911435f-fda0-481b-9492-6747a3045406 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:08.807856 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0911435f-fda0-481b-9492-6747a3045406 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:08.810234 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0911435f-fda0-481b-9492-6747a3045406 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:08.818195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 463/929] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 17:22:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:08.824094 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911435f-fda0-481b-9492-6747a3045406 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:08.840855 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:08.861488 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2ef18-15ee-48f1-ae4c-c69de68537b2 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:08.899278 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:08.909319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3762ad9-d4f2-4900-8eb4-2b138de42400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:08.910510 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3762ad9-d4f2-4900-8eb4-2b138de42400 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:08.911210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3762ad9-d4f2-4900-8eb4-2b138de42400 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:08.911562 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3762ad9-d4f2-4900-8eb4-2b138de42400 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:08.912593 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3762ad9-d4f2-4900-8eb4-2b138de42400 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:08.914733 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3762ad9-d4f2-4900-8eb4-2b138de42400 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:08.929260 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 467/930] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jun 16 17:22:08 2022] GET /identity/v3/services => generated 1497 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:22:08.963364 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911435f-fda0-481b-9492-6747a3045406 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:09.002535 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97453171-9a64-43f5-9d8a-b4101340f7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:09.003225 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97453171-9a64-43f5-9d8a-b4101340f7c2 None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:09.003516 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97453171-9a64-43f5-9d8a-b4101340f7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:09.003801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97453171-9a64-43f5-9d8a-b4101340f7c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:09.004574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97453171-9a64-43f5-9d8a-b4101340f7c2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:09.006033 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97453171-9a64-43f5-9d8a-b4101340f7c2 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:09.033517 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 464/931] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jun 16 17:22:08 2022] POST /identity/v3/endpoints => generated 347 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:09.558863 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ee7b5-5cf9-490c-90ee-e9770afee289 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:09.559199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ee7b5-5cf9-490c-90ee-e9770afee289 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:09.559432 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ee7b5-5cf9-490c-90ee-e9770afee289 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:09.561089 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 468/932] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:09.567274 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9b53d-39e8-42ce-8e93-a14514cf7fc7 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:09.567576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9b53d-39e8-42ce-8e93-a14514cf7fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:09.567863 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9b53d-39e8-42ce-8e93-a14514cf7fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:09.585357 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-4ca9b53d-39e8-42ce-8e93-a14514cf7fc7 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:09.586833 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4ca9b53d-39e8-42ce-8e93-a14514cf7fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:09.653325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:09.667731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 465/933] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:09.691442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ee7b5-5cf9-490c-90ee-e9770afee289 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:09.802306 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ee7b5-5cf9-490c-90ee-e9770afee289 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:09.838792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8dd84-309f-4b60-879e-eccf60b72b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:09.839874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8dd84-309f-4b60-879e-eccf60b72b5d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:09.840310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8dd84-309f-4b60-879e-eccf60b72b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:09.840744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8dd84-309f-4b60-879e-eccf60b72b5d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:09.853589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e8dd84-309f-4b60-879e-eccf60b72b5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:22:09.862957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-34e8dd84-309f-4b60-879e-eccf60b72b5d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:22:09.862957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:09.862957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:09.862957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:09.862957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:09.862957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:09.862957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:09.862957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:09.862957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:09.862957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:09.862957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:09.862957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:09.862957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:22:09.862957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:22:09.862957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:22:09.862957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:22:09.864191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:22:09.864191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:22:09.864191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:22:09.864191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:22:09.864191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:09.864191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:09.864191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:09.864191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:09.864191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:09.864191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:09.864191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:09.864191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:09.864191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:09.864191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:09.864191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:09.864191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:09.865318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:09.865318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:09.865318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:09.865318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:09.865318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:22:09.865318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:22:09.865318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:22:09.865318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:22:09.865318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:22:09.865318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:22:09.865318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:22:09.865318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:09.867931 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 469/934] 10.222.0.40 () {56 vars in 1120 bytes} [Thu Jun 16 17:22:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:09.873726 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9b53d-39e8-42ce-8e93-a14514cf7fc7 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:09.979649 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9b53d-39e8-42ce-8e93-a14514cf7fc7 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:10.015933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932dd6c-5da3-40f0-a96e-7431d268edb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:10.016620 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932dd6c-5da3-40f0-a96e-7431d268edb0 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:10.016923 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932dd6c-5da3-40f0-a96e-7431d268edb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:10.017237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932dd6c-5da3-40f0-a96e-7431d268edb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:10.018005 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932dd6c-5da3-40f0-a96e-7431d268edb0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:10.018322 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932dd6c-5da3-40f0-a96e-7431d268edb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:10.019702 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932dd6c-5da3-40f0-a96e-7431d268edb0 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:10.028659 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 466/935] 10.222.0.40 () {56 vars in 1115 bytes} [Thu Jun 16 17:22:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:10.034814 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8dd84-309f-4b60-879e-eccf60b72b5d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:10.100364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:10.153026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8dd84-309f-4b60-879e-eccf60b72b5d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:10.192533 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc427c2-72c6-4421-a72a-91ae2fde53c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:10.193654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc427c2-72c6-4421-a72a-91ae2fde53c5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:10.194091 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc427c2-72c6-4421-a72a-91ae2fde53c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:10.194532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc427c2-72c6-4421-a72a-91ae2fde53c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:10.207791 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc427c2-72c6-4421-a72a-91ae2fde53c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:10.209530 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc427c2-72c6-4421-a72a-91ae2fde53c5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:10.228826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-0dc427c2-72c6-4421-a72a-91ae2fde53c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:10.228826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:10.228826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:10.228826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:10.228826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:10.228826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:10.228826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:10.228826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:10.228826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:10.228826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:10.228826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:10.228826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:10.228826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:22:10.228826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:22:10.228826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:22:10.230031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:22:10.230031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:10.230031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:10.230031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:10.230031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:10.230031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:10.230031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:10.230031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:10.230031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:10.230031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:10.230031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:10.230031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:10.230031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:10.230031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:10.230031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:10.230031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:10.230031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:10.231150 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:22:10.231150 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:22:10.231150 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:10.231150 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:10.233539 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 470/936] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:10.251532 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932dd6c-5da3-40f0-a96e-7431d268edb0 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:10.317407 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6416e-a3b4-4fc9-be91-090d2465dc97 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:10.317407 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6416e-a3b4-4fc9-be91-090d2465dc97 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:10.317777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6416e-a3b4-4fc9-be91-090d2465dc97 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:10.319326 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 471/937] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:22:10.324165 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afa6e6-6115-4cde-be9c-ccd2aa1e2fbe None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:10.324462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afa6e6-6115-4cde-be9c-ccd2aa1e2fbe None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:10.325042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afa6e6-6115-4cde-be9c-ccd2aa1e2fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:10.337565 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:10.345207 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a5afa6e6-6115-4cde-be9c-ccd2aa1e2fbe None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:10.346849 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a5afa6e6-6115-4cde-be9c-ccd2aa1e2fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:10.387181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932dd6c-5da3-40f0-a96e-7431d268edb0 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:10.425724 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac38c9-65be-4267-9d3a-d528624237ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:10.426401 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac38c9-65be-4267-9d3a-d528624237ff None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:10.426694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac38c9-65be-4267-9d3a-d528624237ff None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:10.426978 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac38c9-65be-4267-9d3a-d528624237ff None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:10.427710 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ac38c9-65be-4267-9d3a-d528624237ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:10.428027 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ac38c9-65be-4267-9d3a-d528624237ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:10.429474 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ac38c9-65be-4267-9d3a-d528624237ff None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:10.430573 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 472/938] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:10.453019 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 467/939] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:10.455560 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afa6e6-6115-4cde-be9c-ccd2aa1e2fbe None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:10.458714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac38c9-65be-4267-9d3a-d528624237ff None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:10.496712 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:10.563839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afa6e6-6115-4cde-be9c-ccd2aa1e2fbe None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:10.567220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac38c9-65be-4267-9d3a-d528624237ff None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:10.582662 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:10.603769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a1c90-8cc2-48a2-828a-a219ec792a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:10.604593 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef35037-cbb1-410b-8455-2fbaadb8e0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:10.605444 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef35037-cbb1-410b-8455-2fbaadb8e0ad None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:10.605741 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef35037-cbb1-410b-8455-2fbaadb8e0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:10.606025 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef35037-cbb1-410b-8455-2fbaadb8e0ad None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:10.606294 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a1c90-8cc2-48a2-828a-a219ec792a41 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:10.606990 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a1c90-8cc2-48a2-828a-a219ec792a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:10.607665 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a1c90-8cc2-48a2-828a-a219ec792a41 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:10.609357 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a1c90-8cc2-48a2-828a-a219ec792a41 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:10.612005 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a1c90-8cc2-48a2-828a-a219ec792a41 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:10.613784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef35037-cbb1-410b-8455-2fbaadb8e0ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:10.614895 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: WARNING keystone.server.flask.common [None req-053a1c90-8cc2-48a2-828a-a219ec792a41 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 17:22:10.615203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef35037-cbb1-410b-8455-2fbaadb8e0ad None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:10.633352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-8ef35037-cbb1-410b-8455-2fbaadb8e0ad None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:22:10.633352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:10.633352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:10.633352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:10.633352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:10.633352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:10.633352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:10.633352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:10.633352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:10.633352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:10.633352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:10.633352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:10.633352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:22:10.633352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:22:10.633352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:22:10.633352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:22:10.635220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:10.635220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:10.635220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:22:10.635220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:22:10.635220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:22:10.635220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:22:10.635220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:10.635220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:10.635220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:10.635220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:10.635220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:10.635220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:10.635220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:10.635220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:10.635220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:10.635220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:10.635220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:10.635220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:10.636671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:10.636671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:10.636671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:22:10.636671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:22:10.636671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:22:10.636671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:22:10.636671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:22:10.636671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:22:10.636671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:22:10.636671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:10.640738 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 468/940] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jun 16 17:22:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:10.641972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef35037-cbb1-410b-8455-2fbaadb8e0ad None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:10.647758 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.sql.core [None req-053a1c90-8cc2-48a2-828a-a219ec792a41 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 17:22:10.647758 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 17:22:10.647758 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [parameters: {'id': '353488387bee4539a3b6d869c89c806e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 17:22:10.647758 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80151) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 17:22:10.658092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-053a1c90-8cc2-48a2-828a-a219ec792a41 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 353488387bee4539a3b6d869c89c806e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 353488387bee4539a3b6d869c89c806e. Jun 16 17:22:10.658092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:10.658092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:22:10.658092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:22:10.658092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:22:10.658092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:22:10.658092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:22:10.658092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:22:10.658092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:22:10.658092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:22:10.658092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:22:10.658092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:22:10.658092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:22:10.658979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application result.read() Jun 16 17:22:10.658979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:22:10.658979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:22:10.658979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:22:10.658979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:22:10.658979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:22:10.658979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:22:10.658979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:22:10.658979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:22:10.658979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 17:22:10.658979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:10.658979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 17:22:10.658979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:10.658979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:10.658979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 17:22:10.658979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 17:22:10.658979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 16 17:22:10.658979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 16 17:22:10.659996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:22:10.659996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:22:10.659996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 17:22:10.659996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application yield resource Jun 16 17:22:10.659996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:22:10.659996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:22:10.659996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 17:22:10.659996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 17:22:10.659996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 17:22:10.659996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 17:22:10.659996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 17:22:10.659996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise self.value Jun 16 17:22:10.659996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 17:22:10.659996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 17:22:10.659996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 17:22:10.659996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application session.commit() Jun 16 17:22:10.659996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 17:22:10.660956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 17:22:10.660956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 17:22:10.660956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 17:22:10.660956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 17:22:10.660956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self.session.flush() Jun 16 17:22:10.660956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 17:22:10.660956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 17:22:10.660956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 17:22:10.660956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 17:22:10.660956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 17:22:10.660956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application compat.raise_( Jun 16 17:22:10.660956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:22:10.660956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception Jun 16 17:22:10.660956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 17:22:10.660956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 17:22:10.660956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 17:22:10.660956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 17:22:10.662859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 17:22:10.662859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 17:22:10.662859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 17:22:10.662859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 17:22:10.662859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 17:22:10.662859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 17:22:10.662859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 17:22:10.662859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 17:22:10.662859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 17:22:10.662859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 17:22:10.662859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 17:22:10.662859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 17:22:10.662859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 17:22:10.662859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 17:22:10.662859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 17:22:10.664927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 17:22:10.664927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:22:10.664927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception Jun 16 17:22:10.664927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:22:10.664927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:22:10.664927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:22:10.664927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:22:10.664927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:22:10.664927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:22:10.664927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:22:10.664927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:22:10.664927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:22:10.664927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:22:10.664927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:22:10.664927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application result.read() Jun 16 17:22:10.664927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:22:10.664927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:22:10.667179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:22:10.667179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:22:10.667179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:22:10.667179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:22:10.667179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:22:10.667179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:22:10.667179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 17:22:10.667179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 17:22:10.667179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [parameters: {'id': '353488387bee4539a3b6d869c89c806e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 17:22:10.667179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 17:22:10.667179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:10.667179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:22:10.667179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:10.667179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:10.667179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 16 17:22:10.669173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 16 17:22:10.669173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 17:22:10.669173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 17:22:10.669173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 16 17:22:10.669173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:10.669173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:22:10.669173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:10.669173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:10.669173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:10.669173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:10.669173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:10.669173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:10.669173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:10.669173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:10.669173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:10.669173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:10.669173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:10.672219 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:10.672219 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 16 17:22:10.672219 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 16 17:22:10.672219 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:10.672219 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:10.672219 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 16 17:22:10.672219 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 16 17:22:10.672219 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 353488387bee4539a3b6d869c89c806e. Jun 16 17:22:10.672219 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:10.675041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 473/941] 10.222.0.40 () {60 vars in 1140 bytes} [Thu Jun 16 17:22:10 2022] POST /identity/v3/projects => generated 262 bytes in 220 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:22:10.680843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a1c90-8cc2-48a2-828a-a219ec792a41 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:10.750116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef35037-cbb1-410b-8455-2fbaadb8e0ad None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:10.787356 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54dd75-c4fa-4387-9822-b970f88076a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:10.787989 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54dd75-c4fa-4387-9822-b970f88076a5 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:10.788281 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54dd75-c4fa-4387-9822-b970f88076a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:10.788565 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54dd75-c4fa-4387-9822-b970f88076a5 None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:10.789444 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54dd75-c4fa-4387-9822-b970f88076a5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:10.789763 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54dd75-c4fa-4387-9822-b970f88076a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:10.790442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:10.792106 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54dd75-c4fa-4387-9822-b970f88076a5 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:10.801076 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a1c90-8cc2-48a2-828a-a219ec792a41 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:10.817432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 469/942] 10.222.0.40 () {56 vars in 1149 bytes} [Thu Jun 16 17:22:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:10.819599 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54dd75-c4fa-4387-9822-b970f88076a5 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:10.840635 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0eb238-85a0-4bcb-a443-96e42ebe4b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:10.841850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0eb238-85a0-4bcb-a443-96e42ebe4b0d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:10.842401 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0eb238-85a0-4bcb-a443-96e42ebe4b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:10.843027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0eb238-85a0-4bcb-a443-96e42ebe4b0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:10.851186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0eb238-85a0-4bcb-a443-96e42ebe4b0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:10.853389 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:10.853864 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0eb238-85a0-4bcb-a443-96e42ebe4b0d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:10.872494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-9d0eb238-85a0-4bcb-a443-96e42ebe4b0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:22:10.872494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:10.872494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:10.872494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:10.872494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:10.872494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:10.872494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:10.872494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:10.872494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:10.872494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:10.872494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:10.872494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:10.872494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:22:10.872494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:22:10.872494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:22:10.874066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:22:10.874066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:10.874066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:10.874066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:10.874066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:10.874066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:10.874066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:10.874066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:10.874066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:10.874066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:10.874066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:10.874066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:10.874066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:10.874066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:10.874066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:10.874066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:10.874066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:10.875240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:22:10.875240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:22:10.875240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:22:10.875240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:22:10.875240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:22:10.875240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:22:10.875240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:22:10.875240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:10.877977 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 474/943] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:22:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:22:10.883973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0eb238-85a0-4bcb-a443-96e42ebe4b0d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:10.929385 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54dd75-c4fa-4387-9822-b970f88076a5 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:10.966157 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b2dc7-d8e8-4b09-95a4-a66e101cd890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:10.966969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b2dc7-d8e8-4b09-95a4-a66e101cd890 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:10.967281 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b2dc7-d8e8-4b09-95a4-a66e101cd890 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:10.967571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b2dc7-d8e8-4b09-95a4-a66e101cd890 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:10.979500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b2dc7-d8e8-4b09-95a4-a66e101cd890 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:10.981372 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b2dc7-d8e8-4b09-95a4-a66e101cd890 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:11.001921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-373b2dc7-d8e8-4b09-95a4-a66e101cd890 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:11.001921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:11.001921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:11.001921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:11.001921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:11.001921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:11.001921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:11.001921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:11.001921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:11.001921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:11.001921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:11.001921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:11.001921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:22:11.001921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:22:11.001921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:22:11.003015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:22:11.003015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:11.003015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:11.003015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:11.003015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:11.003015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:11.003015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:11.003015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:11.003015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:11.003015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:11.003015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:11.003015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:11.003015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:11.003015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:11.003015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:11.003015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:11.003015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:11.003964 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:22:11.003964 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:22:11.003964 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:11.003964 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:11.005441 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0eb238-85a0-4bcb-a443-96e42ebe4b0d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:11.008251 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 470/944] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:22:11.009588 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b2dc7-d8e8-4b09-95a4-a66e101cd890 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:11.043141 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211745e-7a1a-4c3f-8f77-963b360106f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:11.044572 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211745e-7a1a-4c3f-8f77-963b360106f4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:11.045234 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211745e-7a1a-4c3f-8f77-963b360106f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:11.046253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211745e-7a1a-4c3f-8f77-963b360106f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:11.047321 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211745e-7a1a-4c3f-8f77-963b360106f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:11.047846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211745e-7a1a-4c3f-8f77-963b360106f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:11.049507 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211745e-7a1a-4c3f-8f77-963b360106f4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:11.077494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 475/945] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 17:22:10 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:11.101644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:11.120356 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b2dc7-d8e8-4b09-95a4-a66e101cd890 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:11.159622 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66106eb8-23d7-49e5-973e-d8552f17da8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:11.160308 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66106eb8-23d7-49e5-973e-d8552f17da8b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:11.160596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66106eb8-23d7-49e5-973e-d8552f17da8b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:11.160890 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66106eb8-23d7-49e5-973e-d8552f17da8b None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:11.161659 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66106eb8-23d7-49e5-973e-d8552f17da8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:11.161972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66106eb8-23d7-49e5-973e-d8552f17da8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:11.163362 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66106eb8-23d7-49e5-973e-d8552f17da8b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:11.176845 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 471/946] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:11.182704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211745e-7a1a-4c3f-8f77-963b360106f4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:11.203357 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:11.293327 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adadfa-ef7d-4ca1-8080-a29b59343061 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:11.293972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adadfa-ef7d-4ca1-8080-a29b59343061 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:11.294916 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adadfa-ef7d-4ca1-8080-a29b59343061 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:11.296352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 472/947] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:11.303420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc8032-be6c-4f0d-adc7-1e4c6b7702db None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:11.303796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc8032-be6c-4f0d-adc7-1e4c6b7702db None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:11.304147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc8032-be6c-4f0d-adc7-1e4c6b7702db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:11.314647 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211745e-7a1a-4c3f-8f77-963b360106f4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:11.322298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-4afc8032-be6c-4f0d-adc7-1e4c6b7702db None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:11.323507 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4afc8032-be6c-4f0d-adc7-1e4c6b7702db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:11.355749 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24902d72-59dc-4bd9-8ebf-f4b0eab91a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:11.356566 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24902d72-59dc-4bd9-8ebf-f4b0eab91a5f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:11.356858 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24902d72-59dc-4bd9-8ebf-f4b0eab91a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:11.357138 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24902d72-59dc-4bd9-8ebf-f4b0eab91a5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:11.365946 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24902d72-59dc-4bd9-8ebf-f4b0eab91a5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:11.367474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24902d72-59dc-4bd9-8ebf-f4b0eab91a5f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:11.381839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-24902d72-59dc-4bd9-8ebf-f4b0eab91a5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:22:11.381839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:11.381839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:11.381839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:11.381839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:11.381839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:11.381839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:11.381839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:11.381839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:11.381839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:11.381839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:11.381839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:11.381839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:22:11.381839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:22:11.381839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:22:11.383105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:22:11.383105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:11.383105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:11.383105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:11.383105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:11.383105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:11.383105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:11.383105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:11.383105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:11.383105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:11.383105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:11.383105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:11.383105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:11.383105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:11.383105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:11.383105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:11.383105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:11.384058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:22:11.384058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:22:11.384058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:22:11.384058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:22:11.384058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:22:11.384058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:22:11.384058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:22:11.384058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:11.384058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 476/948] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:22:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:11.385197 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50076456-da33-4af8-b133-aa6e28d75b6c None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:11.385197 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50076456-da33-4af8-b133-aa6e28d75b6c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:11.385197 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50076456-da33-4af8-b133-aa6e28d75b6c None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:11.386868 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 477/949] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:11.393082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88c19b-95cb-4d05-a0bb-717dc9afbe13 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:11.393670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88c19b-95cb-4d05-a0bb-717dc9afbe13 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:11.394058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88c19b-95cb-4d05-a0bb-717dc9afbe13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:11.407520 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 473/950] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:22:11.413689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-3e88c19b-95cb-4d05-a0bb-717dc9afbe13 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:11.413982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc8032-be6c-4f0d-adc7-1e4c6b7702db None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:11.415436 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3e88c19b-95cb-4d05-a0bb-717dc9afbe13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:11.496736 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 478/951] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:11.497903 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88c19b-95cb-4d05-a0bb-717dc9afbe13 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:11.507634 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:11.517511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:11.528675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc8032-be6c-4f0d-adc7-1e4c6b7702db None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:11.568018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48454d2e-e02e-4966-bc27-bbe504feae2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:11.568709 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48454d2e-e02e-4966-bc27-bbe504feae2a None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:11.569001 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48454d2e-e02e-4966-bc27-bbe504feae2a None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:11.569314 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48454d2e-e02e-4966-bc27-bbe504feae2a None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:11.570112 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48454d2e-e02e-4966-bc27-bbe504feae2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:11.570428 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48454d2e-e02e-4966-bc27-bbe504feae2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:11.571865 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48454d2e-e02e-4966-bc27-bbe504feae2a None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:11.584893 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 474/952] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:22:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:11.590435 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48454d2e-e02e-4966-bc27-bbe504feae2a None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:11.624435 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88c19b-95cb-4d05-a0bb-717dc9afbe13 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:11.667621 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60490c2e-dd41-4a3d-a276-5b00e3469f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:11.668438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60490c2e-dd41-4a3d-a276-5b00e3469f00 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:11.668724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60490c2e-dd41-4a3d-a276-5b00e3469f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:11.669004 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60490c2e-dd41-4a3d-a276-5b00e3469f00 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:11.671270 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60490c2e-dd41-4a3d-a276-5b00e3469f00 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:11.674718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60490c2e-dd41-4a3d-a276-5b00e3469f00 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:11.688692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-60490c2e-dd41-4a3d-a276-5b00e3469f00 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:22:11.688692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:11.688692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:22:11.688692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:22:11.688692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:22:11.688692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:22:11.688692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:22:11.688692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:11.688692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:22:11.688692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:11.688692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:22:11.688692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:11.688692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:11.688692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:11.688692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:11.688692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:11.688692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:11.689815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:11.689815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:11.689815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:11.689815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:11.689815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:11.689815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:11.689815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:22:11.689815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:22:11.689815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:22:11.689815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:22:11.689815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:11.689815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:11.689815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:11.689815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:11.689815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:11.689815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:11.690724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:11.690724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:11.690724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:11.690724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:11.690724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:11.690724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:11.690724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:11.690724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:11.690724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:11.690724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:11.690724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:22:11.690724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:11.690724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:22:11.690724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:11.695266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 479/953] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:22:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:11.696177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60490c2e-dd41-4a3d-a276-5b00e3469f00 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:11.706898 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48454d2e-e02e-4966-bc27-bbe504feae2a None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:11.747937 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658e5dd-d289-4b0c-8bc2-d1a658760106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:11.748698 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658e5dd-d289-4b0c-8bc2-d1a658760106 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:11.748988 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658e5dd-d289-4b0c-8bc2-d1a658760106 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:11.749570 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658e5dd-d289-4b0c-8bc2-d1a658760106 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:11.750241 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658e5dd-d289-4b0c-8bc2-d1a658760106 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:11.751685 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658e5dd-d289-4b0c-8bc2-d1a658760106 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:11.753789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:11.766676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-f658e5dd-d289-4b0c-8bc2-d1a658760106 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:22:11.766676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:11.766676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:22:11.766676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:22:11.766676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:22:11.766676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:22:11.766676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:22:11.766676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:11.766676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:22:11.766676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:11.766676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:22:11.766676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:11.766676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:11.766676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:11.766676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:11.766676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:11.766676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:11.768085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:11.768085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:11.768085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:11.768085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:11.768085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:11.768085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:11.768085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:22:11.768085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:22:11.768085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:22:11.768085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:22:11.768085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:11.768085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:11.768085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:11.768085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:11.768085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:11.768085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:11.769255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:11.769255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:11.769255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:11.769255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:11.769255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:11.769255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:11.769255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:11.769255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:11.769255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:11.769255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:11.769255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:22:11.769255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:11.769255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:22:11.769255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:11.771299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:11.777366 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 475/954] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 17:22:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:11.777366 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658e5dd-d289-4b0c-8bc2-d1a658760106 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:11.820192 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60490c2e-dd41-4a3d-a276-5b00e3469f00 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:11.858989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ffd7e-5ea3-42a0-88ee-e93a04bd038b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:11.860211 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ffd7e-5ea3-42a0-88ee-e93a04bd038b None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:11.860503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ffd7e-5ea3-42a0-88ee-e93a04bd038b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:11.861061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ffd7e-5ea3-42a0-88ee-e93a04bd038b None admin] PATH_INFO: `/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/cfc2f31f654e486d8505d80aad17d17c/roles/3a41c48a11174f87a500cc1d6618ee65` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:11.863182 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ffd7e-5ea3-42a0-88ee-e93a04bd038b None admin] RBAC: Authorizing `identity:create_grant(project_id=35a535a305e740368b7a4d77d0ad22a0, user_id=cfc2f31f654e486d8505d80aad17d17c, role_id=3a41c48a11174f87a500cc1d6618ee65)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:11.864688 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ffd7e-5ea3-42a0-88ee-e93a04bd038b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:11.885981 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658e5dd-d289-4b0c-8bc2-d1a658760106 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:11.889703 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 480/955] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:22:11 2022] PUT /identity/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/cfc2f31f654e486d8505d80aad17d17c/roles/3a41c48a11174f87a500cc1d6618ee65 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:11.890862 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ffd7e-5ea3-42a0-88ee-e93a04bd038b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:11.927632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0f5d2-a138-4bff-b5a2-a2f9cff36a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:11.928339 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0f5d2-a138-4bff-b5a2-a2f9cff36a79 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:11.928677 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0f5d2-a138-4bff-b5a2-a2f9cff36a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:11.928965 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0f5d2-a138-4bff-b5a2-a2f9cff36a79 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:11.929793 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0f5d2-a138-4bff-b5a2-a2f9cff36a79 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:11.930044 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0f5d2-a138-4bff-b5a2-a2f9cff36a79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:11.931603 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0f5d2-a138-4bff-b5a2-a2f9cff36a79 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:11.941756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 476/956] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 17:22:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:11.945813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0f5d2-a138-4bff-b5a2-a2f9cff36a79 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:12.021711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:12.060774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ffd7e-5ea3-42a0-88ee-e93a04bd038b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:12.079947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0f5d2-a138-4bff-b5a2-a2f9cff36a79 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:12.096704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:12.104126 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa63099-049a-4608-9ccf-23b56c329230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:12.105190 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa63099-049a-4608-9ccf-23b56c329230 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:12.105382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa63099-049a-4608-9ccf-23b56c329230 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:12.105623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa63099-049a-4608-9ccf-23b56c329230 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:12.106353 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa63099-049a-4608-9ccf-23b56c329230 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:12.106674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa63099-049a-4608-9ccf-23b56c329230 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:12.108563 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa63099-049a-4608-9ccf-23b56c329230 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:12.118916 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 481/957] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jun 16 17:22:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:12.120740 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa63099-049a-4608-9ccf-23b56c329230 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:12.122285 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf224b-b0b6-4ec8-a780-aa67f5cf4933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:12.123061 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf224b-b0b6-4ec8-a780-aa67f5cf4933 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:12.123359 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf224b-b0b6-4ec8-a780-aa67f5cf4933 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:12.124155 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf224b-b0b6-4ec8-a780-aa67f5cf4933 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:12.125831 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf224b-b0b6-4ec8-a780-aa67f5cf4933 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:12.127123 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf224b-b0b6-4ec8-a780-aa67f5cf4933 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:12.128525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf224b-b0b6-4ec8-a780-aa67f5cf4933 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:12.135029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 477/958] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 17:22:11 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:12.246632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa63099-049a-4608-9ccf-23b56c329230 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:12.287874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f599d4-6e4f-499c-b58d-f1e6b390c64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:12.288572 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f599d4-6e4f-499c-b58d-f1e6b390c64c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:12.288866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f599d4-6e4f-499c-b58d-f1e6b390c64c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:12.289164 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f599d4-6e4f-499c-b58d-f1e6b390c64c None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:12.289914 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f599d4-6e4f-499c-b58d-f1e6b390c64c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:12.290232 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f599d4-6e4f-499c-b58d-f1e6b390c64c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:12.291647 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f599d4-6e4f-499c-b58d-f1e6b390c64c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:12.298831 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 482/959] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jun 16 17:22:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:12.305385 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf224b-b0b6-4ec8-a780-aa67f5cf4933 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:12.418907 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf224b-b0b6-4ec8-a780-aa67f5cf4933 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:12.446598 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:12.456753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea608e09-831c-4501-97d9-a30a10a2885d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:12.457455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea608e09-831c-4501-97d9-a30a10a2885d None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:12.457740 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea608e09-831c-4501-97d9-a30a10a2885d None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:12.458022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea608e09-831c-4501-97d9-a30a10a2885d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:12.458777 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea608e09-831c-4501-97d9-a30a10a2885d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:12.460142 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea608e09-831c-4501-97d9-a30a10a2885d None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:12.481477 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 478/960] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jun 16 17:22:12 2022] POST /identity/v3/endpoints => generated 349 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:13.397670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba888de5-0d69-42d2-9c91-83e43623529b None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:13.398182 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba888de5-0d69-42d2-9c91-83e43623529b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:13.398261 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba888de5-0d69-42d2-9c91-83e43623529b None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:13.399857 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 483/961] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:13.406038 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd4ba2-0dab-4d13-85fa-1428ae60a0c0 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:13.406502 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd4ba2-0dab-4d13-85fa-1428ae60a0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:13.406800 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd4ba2-0dab-4d13-85fa-1428ae60a0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:13.423959 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-6bdd4ba2-0dab-4d13-85fa-1428ae60a0c0 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:13.425142 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-6bdd4ba2-0dab-4d13-85fa-1428ae60a0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:13.535034 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 479/962] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:13.575929 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba888de5-0d69-42d2-9c91-83e43623529b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:13.645823 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:13.726908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba888de5-0d69-42d2-9c91-83e43623529b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:13.779140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb516dbc-7ff4-4e89-b6e2-0cd5d59b2568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:13.779917 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb516dbc-7ff4-4e89-b6e2-0cd5d59b2568 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:13.780997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb516dbc-7ff4-4e89-b6e2-0cd5d59b2568 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:13.781368 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb516dbc-7ff4-4e89-b6e2-0cd5d59b2568 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:13.782119 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb516dbc-7ff4-4e89-b6e2-0cd5d59b2568 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:13.783549 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb516dbc-7ff4-4e89-b6e2-0cd5d59b2568 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:13.800867 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.sql.core [None req-cb516dbc-7ff4-4e89-b6e2-0cd5d59b2568 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 17:22:13.800867 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 17:22:13.800867 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [parameters: {'user_id': 'd1f55554f11c4229abad35b9bac11a61', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 17:22:13.800867 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80151) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 17:22:13.814359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-cb516dbc-7ff4-4e89-b6e2-0cd5d59b2568 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 17:22:13.814359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:13.814359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:22:13.814359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:22:13.814359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:22:13.814359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:22:13.814359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:22:13.814359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:22:13.814359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:22:13.814359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:22:13.814359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:22:13.814359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:22:13.814359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:22:13.814359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application result.read() Jun 16 17:22:13.814359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:22:13.815299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:22:13.815299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:22:13.815299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:22:13.815299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:22:13.815299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:22:13.815299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:22:13.815299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:22:13.815299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 17:22:13.815299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:13.815299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 17:22:13.815299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:13.815299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:13.815299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 17:22:13.815299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 17:22:13.815299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 16 17:22:13.815299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 16 17:22:13.815299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:22:13.815299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:22:13.816299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 17:22:13.816299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application yield resource Jun 16 17:22:13.816299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:22:13.816299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:22:13.816299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 17:22:13.816299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 17:22:13.816299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 17:22:13.816299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 17:22:13.816299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 17:22:13.816299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise self.value Jun 16 17:22:13.816299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 17:22:13.816299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 17:22:13.816299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 17:22:13.816299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application session.commit() Jun 16 17:22:13.816299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 17:22:13.816299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 17:22:13.816299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 17:22:13.817319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 17:22:13.817319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 17:22:13.817319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self.session.flush() Jun 16 17:22:13.817319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 17:22:13.817319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 17:22:13.817319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 17:22:13.817319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 17:22:13.817319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 17:22:13.817319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application compat.raise_( Jun 16 17:22:13.817319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:22:13.817319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception Jun 16 17:22:13.817319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 17:22:13.817319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 17:22:13.817319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 17:22:13.817319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 17:22:13.817319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 17:22:13.817319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 17:22:13.819058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 17:22:13.819058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 17:22:13.819058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 16 17:22:13.819058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 16 17:22:13.819058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 17:22:13.819058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 17:22:13.819058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 17:22:13.819058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 17:22:13.819058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 17:22:13.819058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 17:22:13.819058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 17:22:13.819058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 17:22:13.819058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 17:22:13.819058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 17:22:13.819058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:22:13.819058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception Jun 16 17:22:13.819954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:22:13.819954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:22:13.819954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:22:13.819954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:22:13.819954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:22:13.819954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:22:13.819954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:22:13.819954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:22:13.819954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:22:13.819954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:22:13.819954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:22:13.819954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application result.read() Jun 16 17:22:13.819954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:22:13.819954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:22:13.819954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:22:13.819954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:22:13.820846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:22:13.820846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:22:13.820846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:22:13.820846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:22:13.820846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 17:22:13.820846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 17:22:13.820846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd1f55554f11c4229abad35b9bac11a61', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 17:22:13.820846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 17:22:13.820846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:13.820846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:22:13.820846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:13.820846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:13.820846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:13.820846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:13.820846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:13.826178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:13.826178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:13.826178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:13.826178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:13.826178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:13.826178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:13.826178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:13.826178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 16 17:22:13.826178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 16 17:22:13.826178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:13.826178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:13.826178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:22:13.826178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:22:13.826178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:22:13.826178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:22:13.826178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 16 17:22:13.826178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 16 17:22:13.829064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 16 17:22:13.829064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 16 17:22:13.829064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 17:22:13.829064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 17:22:13.829064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 17:22:13.829064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:13.829064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 484/963] 10.222.0.40 () {60 vars in 1131 bytes} [Thu Jun 16 17:22:13 2022] POST /identity/v3/users => generated 160 bytes in 246 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:13.830104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd4ba2-0dab-4d13-85fa-1428ae60a0c0 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:13.991841 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd4ba2-0dab-4d13-85fa-1428ae60a0c0 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:14.009137 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:14.066010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622fb51-2c20-45e4-817e-a1208bdeeab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:14.066746 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622fb51-2c20-45e4-817e-a1208bdeeab1 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:14.067039 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622fb51-2c20-45e4-817e-a1208bdeeab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:14.067324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622fb51-2c20-45e4-817e-a1208bdeeab1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:14.079316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622fb51-2c20-45e4-817e-a1208bdeeab1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:14.080775 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622fb51-2c20-45e4-817e-a1208bdeeab1 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:14.108293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-d622fb51-2c20-45e4-817e-a1208bdeeab1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:22:14.108293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:14.108293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:14.108293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:14.108293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:14.108293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:14.108293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:14.108293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:14.108293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:14.108293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:14.108293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:14.108293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:14.108293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:22:14.108293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:22:14.108293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:22:14.108293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:22:14.109397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:14.109397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:14.109397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:22:14.109397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:22:14.109397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:22:14.109397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:22:14.109397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:14.109397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:14.109397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:14.109397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:14.109397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:14.109397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:14.109397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:14.109397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:14.109397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:14.109397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:14.109397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:14.109397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:14.110466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:14.110466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:14.110466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:22:14.110466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:22:14.110466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:22:14.110466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:22:14.110466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:22:14.110466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:22:14.110466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:22:14.110466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:14.121309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb516dbc-7ff4-4e89-b6e2-0cd5d59b2568 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:14.128832 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 480/964] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:22:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:14.198539 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc70af7-7648-4e9c-8022-2beab8605680 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:14.198735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc70af7-7648-4e9c-8022-2beab8605680 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:14.199024 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc70af7-7648-4e9c-8022-2beab8605680 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:14.200663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 481/965] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:22:14.205991 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa0ab0-3126-4a8d-bb30-334aec87d0bc None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:14.206436 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa0ab0-3126-4a8d-bb30-334aec87d0bc None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:14.206662 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa0ab0-3126-4a8d-bb30-334aec87d0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:14.238791 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-56fa0ab0-3126-4a8d-bb30-334aec87d0bc None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:14.241117 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-56fa0ab0-3126-4a8d-bb30-334aec87d0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:14.274687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb516dbc-7ff4-4e89-b6e2-0cd5d59b2568 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:14.294777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:14.318112 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a6f32-867a-4544-b172-a651bdcf1318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:14.318653 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a6f32-867a-4544-b172-a651bdcf1318 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:14.319041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a6f32-867a-4544-b172-a651bdcf1318 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:14.319515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a6f32-867a-4544-b172-a651bdcf1318 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:14.320365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a6f32-867a-4544-b172-a651bdcf1318 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:14.320784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a6f32-867a-4544-b172-a651bdcf1318 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:14.322679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a6f32-867a-4544-b172-a651bdcf1318 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:14.349627 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 482/966] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:22:14.365087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 485/967] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 17:22:14 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:14.368191 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa0ab0-3126-4a8d-bb30-334aec87d0bc None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:14.485618 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:14.489858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa0ab0-3126-4a8d-bb30-334aec87d0bc None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:14.529666 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38332e8-2014-4f78-8fb4-1a4066466f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:14.531265 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38332e8-2014-4f78-8fb4-1a4066466f1d None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:14.531562 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38332e8-2014-4f78-8fb4-1a4066466f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:14.531844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38332e8-2014-4f78-8fb4-1a4066466f1d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:14.542546 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b09a0-99f5-4815-88c6-4d3f45fad97b None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:14.543475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b09a0-99f5-4815-88c6-4d3f45fad97b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:14.543933 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b09a0-99f5-4815-88c6-4d3f45fad97b None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:14.544171 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38332e8-2014-4f78-8fb4-1a4066466f1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:22:14.550126 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 486/968] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:14.552149 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b00d49-1c98-4fc2-a5fa-e9578af2c3a4 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:14.552652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b00d49-1c98-4fc2-a5fa-e9578af2c3a4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:14.553144 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b00d49-1c98-4fc2-a5fa-e9578af2c3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:14.553286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-c38332e8-2014-4f78-8fb4-1a4066466f1d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:22:14.553286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:14.553286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:14.553286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:14.553286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:14.553286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:14.553286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:14.553286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:14.553286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:14.553286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:14.553286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:14.553286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:14.553286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:22:14.553286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:22:14.553286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:22:14.553286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:22:14.554235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:22:14.554235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:22:14.554235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:22:14.554235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:22:14.554235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:14.554235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:14.554235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:14.554235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:14.554235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:14.554235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:14.554235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:14.554235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:14.554235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:14.554235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:14.554235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:14.554235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:14.555703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:14.555703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:14.555703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:14.555703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:14.555703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:22:14.555703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:22:14.555703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:22:14.555703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:22:14.555703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:22:14.555703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:22:14.555703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:22:14.555703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:14.555703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 483/969] 10.222.0.40 () {56 vars in 1120 bytes} [Thu Jun 16 17:22:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:14.560486 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38332e8-2014-4f78-8fb4-1a4066466f1d None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:14.581262 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-12b00d49-1c98-4fc2-a5fa-e9578af2c3a4 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:14.584117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-12b00d49-1c98-4fc2-a5fa-e9578af2c3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:14.707228 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 487/970] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:14.709837 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38332e8-2014-4f78-8fb4-1a4066466f1d None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:14.728619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57e15f-7feb-433c-91d3-e0c44961c5f2 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:14.728926 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57e15f-7feb-433c-91d3-e0c44961c5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:14.729281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57e15f-7feb-433c-91d3-e0c44961c5f2 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:14.730627 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 488/971] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:14.731313 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57e15f-7feb-433c-91d3-e0c44961c5f2 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:14.752255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730c857-42a8-4a3b-9f78-efbd7d2579a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:14.752932 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730c857-42a8-4a3b-9f78-efbd7d2579a3 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:14.753255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730c857-42a8-4a3b-9f78-efbd7d2579a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:14.753593 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730c857-42a8-4a3b-9f78-efbd7d2579a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:14.754356 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1730c857-42a8-4a3b-9f78-efbd7d2579a3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:14.754672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1730c857-42a8-4a3b-9f78-efbd7d2579a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:14.756043 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1730c857-42a8-4a3b-9f78-efbd7d2579a3 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:14.764840 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 484/972] 10.222.0.40 () {56 vars in 1115 bytes} [Thu Jun 16 17:22:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:14.766423 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9f7b5-ab99-4d65-9317-7fb3868cda3c None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:14.767007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9f7b5-ab99-4d65-9317-7fb3868cda3c None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:14.767538 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9f7b5-ab99-4d65-9317-7fb3868cda3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:14.783886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-d7c9f7b5-ab99-4d65-9317-7fb3868cda3c None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:14.785375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-d7c9f7b5-ab99-4d65-9317-7fb3868cda3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:14.800457 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:14.845461 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57e15f-7feb-433c-91d3-e0c44961c5f2 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:14.852846 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 485/973] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:14.853659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:14.853778 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9f7b5-ab99-4d65-9317-7fb3868cda3c None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:14.886141 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14687d-e33b-42bf-aeeb-43a1bdc207bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:14.886814 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14687d-e33b-42bf-aeeb-43a1bdc207bb None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:14.887101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14687d-e33b-42bf-aeeb-43a1bdc207bb None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:14.887383 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14687d-e33b-42bf-aeeb-43a1bdc207bb None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:14.888109 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e14687d-e33b-42bf-aeeb-43a1bdc207bb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:14.889613 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e14687d-e33b-42bf-aeeb-43a1bdc207bb None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:14.902274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 489/974] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 17:22:14 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:14.903062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14687d-e33b-42bf-aeeb-43a1bdc207bb None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:14.971384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9f7b5-ab99-4d65-9317-7fb3868cda3c None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:15.024045 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d6898-9b83-415e-b90c-9d6e87e03717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:15.025359 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d6898-9b83-415e-b90c-9d6e87e03717 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:15.025792 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d6898-9b83-415e-b90c-9d6e87e03717 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:15.026196 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d6898-9b83-415e-b90c-9d6e87e03717 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:15.028538 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14687d-e33b-42bf-aeeb-43a1bdc207bb None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:15.044215 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d6898-9b83-415e-b90c-9d6e87e03717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:15.047510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d6898-9b83-415e-b90c-9d6e87e03717 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:15.056680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:15.071122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-a04d6898-9b83-415e-b90c-9d6e87e03717 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:15.071122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:15.071122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:15.071122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:15.071122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:15.071122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:15.071122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:15.071122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:15.071122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:15.071122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:15.071122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:15.071122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:15.071122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:22:15.071122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:22:15.071122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:22:15.072118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:22:15.072118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:15.072118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:15.072118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:15.072118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:15.072118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:15.072118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:15.072118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:15.072118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:15.072118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:15.072118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:15.072118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:15.072118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:15.072118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:15.072118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:15.072118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:15.072118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:15.073063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:22:15.073063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:22:15.073063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:15.073063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:15.073063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 486/975] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:15.078806 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d6898-9b83-415e-b90c-9d6e87e03717 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:15.080925 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb818e-1d99-4350-b0f9-6f75714ad923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:15.081892 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb818e-1d99-4350-b0f9-6f75714ad923 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:15.082181 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb818e-1d99-4350-b0f9-6f75714ad923 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:15.082457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb818e-1d99-4350-b0f9-6f75714ad923 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:15.083895 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb818e-1d99-4350-b0f9-6f75714ad923 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:15.084538 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb818e-1d99-4350-b0f9-6f75714ad923 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:15.099771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-e1bb818e-1d99-4350-b0f9-6f75714ad923 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:22:15.099771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:15.099771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:22:15.099771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:22:15.099771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:22:15.099771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:22:15.099771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:22:15.099771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:15.099771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:22:15.099771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:15.099771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:22:15.099771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:15.099771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:15.099771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:15.099771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:15.099771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:15.099771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:15.100808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:15.100808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:15.100808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:15.100808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:15.100808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:15.100808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:15.100808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:22:15.100808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:22:15.100808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:22:15.100808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:22:15.100808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:15.100808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:15.100808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:15.100808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:15.100808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:15.100808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:15.101768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:15.101768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:15.101768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:15.101768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:15.101768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:15.101768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:15.101768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:15.101768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:15.101768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:15.101768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:15.101768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:22:15.101768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:15.101768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:22:15.101768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:15.102545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 490/976] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 17:22:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:15.109079 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb818e-1d99-4350-b0f9-6f75714ad923 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:15.139954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:15.193242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d6898-9b83-415e-b90c-9d6e87e03717 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:15.223149 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb818e-1d99-4350-b0f9-6f75714ad923 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:15.234797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998dccf-e8d0-469d-8a75-af3a402d7a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:15.235487 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998dccf-e8d0-469d-8a75-af3a402d7a5c None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:15.235779 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998dccf-e8d0-469d-8a75-af3a402d7a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:15.236062 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998dccf-e8d0-469d-8a75-af3a402d7a5c None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:15.238875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998dccf-e8d0-469d-8a75-af3a402d7a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:15.239218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998dccf-e8d0-469d-8a75-af3a402d7a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:15.240612 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998dccf-e8d0-469d-8a75-af3a402d7a5c None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:15.252900 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 487/977] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:15.258660 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998dccf-e8d0-469d-8a75-af3a402d7a5c None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:15.272570 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6704df7-14d6-454e-9e7e-68d8f0bd58f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:15.273379 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6704df7-14d6-454e-9e7e-68d8f0bd58f9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:15.273732 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6704df7-14d6-454e-9e7e-68d8f0bd58f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:15.274078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6704df7-14d6-454e-9e7e-68d8f0bd58f9 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:15.274870 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6704df7-14d6-454e-9e7e-68d8f0bd58f9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:15.275268 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6704df7-14d6-454e-9e7e-68d8f0bd58f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:15.276669 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6704df7-14d6-454e-9e7e-68d8f0bd58f9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:15.283898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 491/978] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jun 16 17:22:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:15.289496 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6704df7-14d6-454e-9e7e-68d8f0bd58f9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:15.325492 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:15.379789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998dccf-e8d0-469d-8a75-af3a402d7a5c None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:15.394281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:15.405306 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6704df7-14d6-454e-9e7e-68d8f0bd58f9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:15.419940 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fcdd47-8375-4a7a-a2d8-0ed28ca9ae4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:15.421630 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcdd47-8375-4a7a-a2d8-0ed28ca9ae4c None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:15.421923 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcdd47-8375-4a7a-a2d8-0ed28ca9ae4c None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:15.422210 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcdd47-8375-4a7a-a2d8-0ed28ca9ae4c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:15.430222 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fcdd47-8375-4a7a-a2d8-0ed28ca9ae4c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:15.432116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fcdd47-8375-4a7a-a2d8-0ed28ca9ae4c None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:15.442683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b4a32-ee45-4dd7-a65a-072ba91778e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:15.443358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b4a32-ee45-4dd7-a65a-072ba91778e9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:15.443657 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b4a32-ee45-4dd7-a65a-072ba91778e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:15.443927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b4a32-ee45-4dd7-a65a-072ba91778e9 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:15.444635 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b4a32-ee45-4dd7-a65a-072ba91778e9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:15.444946 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b4a32-ee45-4dd7-a65a-072ba91778e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:15.446343 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b4a32-ee45-4dd7-a65a-072ba91778e9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:15.447261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-a3fcdd47-8375-4a7a-a2d8-0ed28ca9ae4c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:22:15.447261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:15.447261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:15.447261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:15.447261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:15.447261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:15.447261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:15.447261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:15.447261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:15.447261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:15.447261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:15.447261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:15.447261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:22:15.447261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:22:15.447261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:22:15.447261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:22:15.448203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:15.448203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:15.448203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:22:15.448203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:22:15.448203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:22:15.448203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:22:15.448203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:15.448203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:15.448203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:15.448203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:15.448203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:15.448203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:15.448203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:15.448203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:15.448203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:15.448203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:15.448203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:15.448203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:15.449205 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:15.449205 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:15.449205 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:22:15.449205 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:22:15.449205 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:22:15.449205 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:22:15.449205 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:22:15.449205 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:22:15.449205 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:22:15.449205 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:15.449205 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 488/979] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jun 16 17:22:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:22:15.456012 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fcdd47-8375-4a7a-a2d8-0ed28ca9ae4c None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:15.459419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 492/980] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jun 16 17:22:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:15.466422 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b4a32-ee45-4dd7-a65a-072ba91778e9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:15.566342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fcdd47-8375-4a7a-a2d8-0ed28ca9ae4c None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:15.579949 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b4a32-ee45-4dd7-a65a-072ba91778e9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:15.585174 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:15.606487 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04f0da-8086-4a56-bd7f-8631f1780ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:15.607321 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04f0da-8086-4a56-bd7f-8631f1780ada None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:15.607635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04f0da-8086-4a56-bd7f-8631f1780ada None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:15.607926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04f0da-8086-4a56-bd7f-8631f1780ada None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:15.608927 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04f0da-8086-4a56-bd7f-8631f1780ada None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:15.609272 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04f0da-8086-4a56-bd7f-8631f1780ada None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:15.610702 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04f0da-8086-4a56-bd7f-8631f1780ada None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:15.618325 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6257570-b9a1-49cb-a381-3458b7263287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:15.618996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6257570-b9a1-49cb-a381-3458b7263287 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:15.619281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6257570-b9a1-49cb-a381-3458b7263287 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:15.619564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6257570-b9a1-49cb-a381-3458b7263287 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:15.620682 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6257570-b9a1-49cb-a381-3458b7263287 None admin] RBAC: Adding query filter params (interface=admin, service_id=a1d7529d66a44838989ce4e5b6f3c307, region_id=RegionOne) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:15.620682 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6257570-b9a1-49cb-a381-3458b7263287 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:15.622089 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6257570-b9a1-49cb-a381-3458b7263287 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:15.632504 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 493/981] 10.222.0.40 () {56 vars in 1249 bytes} [Thu Jun 16 17:22:15 2022] GET /identity/v3/endpoints?service_id=a1d7529d66a44838989ce4e5b6f3c307&interface=admin®ion_id=RegionOne => generated 193 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:15.632654 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 489/982] 10.222.0.40 () {56 vars in 1149 bytes} [Thu Jun 16 17:22:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:15.634083 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6257570-b9a1-49cb-a381-3458b7263287 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:15.635383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04f0da-8086-4a56-bd7f-8631f1780ada None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:15.654007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:15.744122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04f0da-8086-4a56-bd7f-8631f1780ada None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:15.744753 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6257570-b9a1-49cb-a381-3458b7263287 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:15.780584 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e4117-6a59-4cac-836e-1b880de228b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:15.781340 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e4117-6a59-4cac-836e-1b880de228b1 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:15.781632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e4117-6a59-4cac-836e-1b880de228b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:15.781937 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e4117-6a59-4cac-836e-1b880de228b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:15.783107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6bd38-2ac0-4c9b-a055-0e0521e82096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:15.783782 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6bd38-2ac0-4c9b-a055-0e0521e82096 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:15.784074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6bd38-2ac0-4c9b-a055-0e0521e82096 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:15.784357 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6bd38-2ac0-4c9b-a055-0e0521e82096 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:15.785049 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a6bd38-2ac0-4c9b-a055-0e0521e82096 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:15.786454 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a6bd38-2ac0-4c9b-a055-0e0521e82096 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:15.793000 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 494/983] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jun 16 17:22:15 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:15.834240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:15.862058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e4117-6a59-4cac-836e-1b880de228b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:15.863568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e4117-6a59-4cac-836e-1b880de228b1 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:15.976270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 490/984] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:22:15 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 17:22:15.981652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6bd38-2ac0-4c9b-a055-0e0521e82096 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:16.072973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:16.093615 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6bd38-2ac0-4c9b-a055-0e0521e82096 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:16.128547 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469606d2-9113-4a2f-8787-9735e182c3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:16.129541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469606d2-9113-4a2f-8787-9735e182c3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:16.129833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469606d2-9113-4a2f-8787-9735e182c3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:16.130110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469606d2-9113-4a2f-8787-9735e182c3d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:16.142457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469606d2-9113-4a2f-8787-9735e182c3d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:16.143916 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469606d2-9113-4a2f-8787-9735e182c3d0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:16.161503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-469606d2-9113-4a2f-8787-9735e182c3d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:16.161503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:16.161503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:16.161503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:16.161503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:16.161503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:16.161503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:16.161503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:16.161503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:16.161503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:16.161503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:16.161503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:16.161503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:22:16.161503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:22:16.161503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:22:16.162477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:22:16.162477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:16.162477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:16.162477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:16.162477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:16.162477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:16.162477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:16.162477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:16.162477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:16.162477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:16.162477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:16.162477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:16.162477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:16.162477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:16.162477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:16.162477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:16.162477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:16.163415 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:22:16.163415 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:22:16.163415 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:16.163415 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:16.163415 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 495/985] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:16.168263 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e4117-6a59-4cac-836e-1b880de228b1 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:16.258164 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:16.282556 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e4117-6a59-4cac-836e-1b880de228b1 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:16.318702 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46577a-9911-4c01-bc6e-e0322681bb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:16.319374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46577a-9911-4c01-bc6e-e0322681bb00 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:16.319661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46577a-9911-4c01-bc6e-e0322681bb00 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:16.319942 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46577a-9911-4c01-bc6e-e0322681bb00 None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:16.320673 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46577a-9911-4c01-bc6e-e0322681bb00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:16.320981 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46577a-9911-4c01-bc6e-e0322681bb00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:16.322383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46577a-9911-4c01-bc6e-e0322681bb00 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:16.338888 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 491/986] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:16.340738 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469606d2-9113-4a2f-8787-9735e182c3d0 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:16.443840 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469606d2-9113-4a2f-8787-9735e182c3d0 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:16.477358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c82f57-06c7-4857-9ee0-4c85193738e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:16.478166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c82f57-06c7-4857-9ee0-4c85193738e4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:16.478455 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c82f57-06c7-4857-9ee0-4c85193738e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:16.478733 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c82f57-06c7-4857-9ee0-4c85193738e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:16.480763 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:16.488407 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c82f57-06c7-4857-9ee0-4c85193738e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:16.489858 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c82f57-06c7-4857-9ee0-4c85193738e4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:16.505088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-89c82f57-06c7-4857-9ee0-4c85193738e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:22:16.505088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:16.505088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:16.505088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:16.505088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:16.505088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:16.505088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:16.505088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:16.505088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:16.505088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:16.505088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:16.505088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:16.505088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:22:16.505088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:22:16.505088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:22:16.506171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:22:16.506171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:16.506171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:16.506171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:16.506171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:16.506171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:16.506171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:16.506171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:16.506171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:16.506171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:16.506171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:16.506171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:16.506171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:16.506171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:16.506171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:16.506171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:16.506171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:16.507108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:22:16.507108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:22:16.507108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:22:16.507108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:22:16.507108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:22:16.507108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:22:16.507108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:22:16.507108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:16.509469 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 496/987] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:22:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:16.512412 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46577a-9911-4c01-bc6e-e0322681bb00 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:16.615936 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46577a-9911-4c01-bc6e-e0322681bb00 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:16.650294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fea490-09df-4f1a-8bca-39a280f7982e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:16.650967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fea490-09df-4f1a-8bca-39a280f7982e None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:16.651261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fea490-09df-4f1a-8bca-39a280f7982e None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:16.651644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fea490-09df-4f1a-8bca-39a280f7982e None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:16.652470 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fea490-09df-4f1a-8bca-39a280f7982e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:16.652795 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fea490-09df-4f1a-8bca-39a280f7982e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:16.654259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fea490-09df-4f1a-8bca-39a280f7982e None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:16.657539 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:16.673181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 492/988] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:22:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:16.675758 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c82f57-06c7-4857-9ee0-4c85193738e4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:16.777620 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c82f57-06c7-4857-9ee0-4c85193738e4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:16.813173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf247ee-dde6-4d00-84ab-dd45055571ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:16.813857 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf247ee-dde6-4d00-84ab-dd45055571ac None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:16.814145 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf247ee-dde6-4d00-84ab-dd45055571ac None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:16.814422 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf247ee-dde6-4d00-84ab-dd45055571ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:16.815260 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf247ee-dde6-4d00-84ab-dd45055571ac None admin] RBAC: Adding query filter params (role.id=3a41c48a11174f87a500cc1d6618ee65, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=cfc2f31f654e486d8505d80aad17d17c) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:16.815579 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf247ee-dde6-4d00-84ab-dd45055571ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:16.817003 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf247ee-dde6-4d00-84ab-dd45055571ac None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:16.829505 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 497/989] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:22:16 2022] GET /identity/v3/role_assignments?user.id=cfc2f31f654e486d8505d80aad17d17c&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=3a41c48a11174f87a500cc1d6618ee65 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:17.147405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce424d9-ac02-4a23-89b0-6e5b88017c68 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:17.149816 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce424d9-ac02-4a23-89b0-6e5b88017c68 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:17.149816 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce424d9-ac02-4a23-89b0-6e5b88017c68 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:17.149816 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 493/990] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:17.155082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c87968-85b8-4ef9-9937-99f43faa279c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:17.155992 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c87968-85b8-4ef9-9937-99f43faa279c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:17.156285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c87968-85b8-4ef9-9937-99f43faa279c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:17.173043 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-48c87968-85b8-4ef9-9937-99f43faa279c None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:17.174274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-48c87968-85b8-4ef9-9937-99f43faa279c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:17.233363 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:17.252604 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 498/991] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:17.277241 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce424d9-ac02-4a23-89b0-6e5b88017c68 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:17.391069 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce424d9-ac02-4a23-89b0-6e5b88017c68 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:17.428095 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d9134-5194-4ab8-b025-92cffb5ee350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:17.428866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d9134-5194-4ab8-b025-92cffb5ee350 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:17.429289 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d9134-5194-4ab8-b025-92cffb5ee350 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:17.429578 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d9134-5194-4ab8-b025-92cffb5ee350 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:17.430243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d9134-5194-4ab8-b025-92cffb5ee350 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:17.431611 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d9134-5194-4ab8-b025-92cffb5ee350 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:17.444450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-cd9d9134-5194-4ab8-b025-92cffb5ee350 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:22:17.444450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:17.444450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:22:17.444450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:22:17.444450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:22:17.444450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:22:17.444450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:22:17.444450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:17.444450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:22:17.444450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:17.444450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:22:17.444450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:17.444450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:17.444450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:17.444450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:17.444450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:17.444450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:17.445774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:17.445774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:17.445774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:17.445774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:17.445774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:17.445774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:17.445774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:22:17.445774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:22:17.445774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:22:17.445774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:22:17.445774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:17.445774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:17.445774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:17.445774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:17.445774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:17.445774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:17.446743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:17.446743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:17.446743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:17.446743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:17.446743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:17.446743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:17.446743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:17.446743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:17.446743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:17.446743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:17.446743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:22:17.446743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:17.446743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:22:17.446743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:17.448098 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 494/992] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:22:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:17.452375 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c87968-85b8-4ef9-9937-99f43faa279c None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:17.562183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c87968-85b8-4ef9-9937-99f43faa279c None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:17.599093 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d03c53-0b74-4cc5-b001-ecbe8ddaee54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:17.599776 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d03c53-0b74-4cc5-b001-ecbe8ddaee54 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:17.600069 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d03c53-0b74-4cc5-b001-ecbe8ddaee54 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:17.600352 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d03c53-0b74-4cc5-b001-ecbe8ddaee54 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:17.601079 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d03c53-0b74-4cc5-b001-ecbe8ddaee54 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:17.601426 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d03c53-0b74-4cc5-b001-ecbe8ddaee54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:17.602806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d03c53-0b74-4cc5-b001-ecbe8ddaee54 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:17.613264 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 499/993] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 17:22:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:17.614629 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d9134-5194-4ab8-b025-92cffb5ee350 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:17.683617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:17.727991 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d9134-5194-4ab8-b025-92cffb5ee350 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:17.762016 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871bd1e-db84-45ff-ac17-02d0cd12ee8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:17.762692 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871bd1e-db84-45ff-ac17-02d0cd12ee8d None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:17.762983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871bd1e-db84-45ff-ac17-02d0cd12ee8d None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:17.763268 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871bd1e-db84-45ff-ac17-02d0cd12ee8d None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:17.764500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871bd1e-db84-45ff-ac17-02d0cd12ee8d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:17.764922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871bd1e-db84-45ff-ac17-02d0cd12ee8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:17.766338 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871bd1e-db84-45ff-ac17-02d0cd12ee8d None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:17.776252 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 495/994] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 17:22:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:17.777503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d03c53-0b74-4cc5-b001-ecbe8ddaee54 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:17.831890 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:17.880765 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d03c53-0b74-4cc5-b001-ecbe8ddaee54 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:17.920130 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066ac00-0ffc-45db-b9cc-baa6e7558f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:17.920985 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066ac00-0ffc-45db-b9cc-baa6e7558f2f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:17.921294 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066ac00-0ffc-45db-b9cc-baa6e7558f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:17.921605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066ac00-0ffc-45db-b9cc-baa6e7558f2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:17.922397 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066ac00-0ffc-45db-b9cc-baa6e7558f2f None admin] RBAC: Adding query filter params (interface=public, service_id=c9f97f24f21048809d7e430e61ff2e4e, region_id=RegionOne) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:17.922718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066ac00-0ffc-45db-b9cc-baa6e7558f2f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:17.924106 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066ac00-0ffc-45db-b9cc-baa6e7558f2f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:17.934273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 500/995] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jun 16 17:22:17 2022] GET /identity/v3/endpoints?service_id=c9f97f24f21048809d7e430e61ff2e4e&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:17.935354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871bd1e-db84-45ff-ac17-02d0cd12ee8d None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:17.960858 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9d2a3-633c-48d5-880f-ff71a336c5a3 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:17.961181 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9d2a3-633c-48d5-880f-ff71a336c5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:17.961463 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9d2a3-633c-48d5-880f-ff71a336c5a3 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:17.962843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 501/996] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:22:17.967703 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f1662-2296-4a20-97ba-eb538a27a7d9 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:17.967997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f1662-2296-4a20-97ba-eb538a27a7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:17.968280 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f1662-2296-4a20-97ba-eb538a27a7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:17.985764 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d80f1662-2296-4a20-97ba-eb538a27a7d9 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:17.987245 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d80f1662-2296-4a20-97ba-eb538a27a7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:18.055482 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871bd1e-db84-45ff-ac17-02d0cd12ee8d None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:18.074029 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 502/997] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:22:18.084440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:18.097408 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f1662-2296-4a20-97ba-eb538a27a7d9 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:18.101745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67cf5b4-6a87-4739-930a-cb00a7fc4939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:18.102425 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cf5b4-6a87-4739-930a-cb00a7fc4939 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:18.102716 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cf5b4-6a87-4739-930a-cb00a7fc4939 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:18.103002 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cf5b4-6a87-4739-930a-cb00a7fc4939 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:18.103698 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67cf5b4-6a87-4739-930a-cb00a7fc4939 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:18.105074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67cf5b4-6a87-4739-930a-cb00a7fc4939 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:18.118651 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 496/998] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jun 16 17:22:17 2022] GET /identity/v3/services => generated 1727 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:22:18.157129 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:18.216041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f1662-2296-4a20-97ba-eb538a27a7d9 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:18.261955 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55240097-dfdc-4157-a9b4-bcefe7d75291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:18.262792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55240097-dfdc-4157-a9b4-bcefe7d75291 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:18.263082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55240097-dfdc-4157-a9b4-bcefe7d75291 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:18.263363 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55240097-dfdc-4157-a9b4-bcefe7d75291 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:18.264081 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55240097-dfdc-4157-a9b4-bcefe7d75291 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:18.265517 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55240097-dfdc-4157-a9b4-bcefe7d75291 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:18.278698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-55240097-dfdc-4157-a9b4-bcefe7d75291 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:22:18.278698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:18.278698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:22:18.278698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:22:18.278698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:22:18.278698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:22:18.278698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:22:18.278698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:18.278698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:22:18.278698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:18.278698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:22:18.278698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:18.278698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:18.278698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:18.278698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:18.278698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:18.278698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:18.280277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:18.280277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:18.280277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:18.280277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:18.280277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:18.280277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:18.280277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:22:18.280277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:22:18.280277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:22:18.280277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:22:18.280277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:18.280277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:18.280277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:18.280277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:18.280277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:18.280277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:18.281202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:18.281202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:18.281202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:18.281202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:18.281202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:18.281202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:18.281202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:18.281202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:18.281202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:18.281202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:18.281202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:22:18.281202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:18.281202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:22:18.281202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:18.281202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 503/999] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 17:22:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:22:18.286678 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67cf5b4-6a87-4739-930a-cb00a7fc4939 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:18.393412 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67cf5b4-6a87-4739-930a-cb00a7fc4939 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:18.428316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd38ddb-229c-42b5-8a2e-60a094b4b136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:18.428983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd38ddb-229c-42b5-8a2e-60a094b4b136 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:18.429301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd38ddb-229c-42b5-8a2e-60a094b4b136 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:18.429584 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd38ddb-229c-42b5-8a2e-60a094b4b136 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:18.430297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd38ddb-229c-42b5-8a2e-60a094b4b136 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:18.430608 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd38ddb-229c-42b5-8a2e-60a094b4b136 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:18.431969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd38ddb-229c-42b5-8a2e-60a094b4b136 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:18.437699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 497/1000] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jun 16 17:22:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:18.443134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55240097-dfdc-4157-a9b4-bcefe7d75291 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:18.543698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55240097-dfdc-4157-a9b4-bcefe7d75291 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:18.555726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:18.580736 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34518358-e5a4-4229-9920-ee97313af4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:18.581728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34518358-e5a4-4229-9920-ee97313af4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:18.582189 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34518358-e5a4-4229-9920-ee97313af4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:18.582615 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34518358-e5a4-4229-9920-ee97313af4f7 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:18.583483 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34518358-e5a4-4229-9920-ee97313af4f7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:18.583947 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34518358-e5a4-4229-9920-ee97313af4f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:18.585508 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34518358-e5a4-4229-9920-ee97313af4f7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:18.591196 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 504/1001] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jun 16 17:22:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:18.598391 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd38ddb-229c-42b5-8a2e-60a094b4b136 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:18.654917 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:18.704666 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd38ddb-229c-42b5-8a2e-60a094b4b136 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:18.738551 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badfd8bc-1fff-4c5f-a4d6-3e58110fe872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:18.739223 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfd8bc-1fff-4c5f-a4d6-3e58110fe872 None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:18.739517 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfd8bc-1fff-4c5f-a4d6-3e58110fe872 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:18.739806 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfd8bc-1fff-4c5f-a4d6-3e58110fe872 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:18.740573 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badfd8bc-1fff-4c5f-a4d6-3e58110fe872 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:18.741982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badfd8bc-1fff-4c5f-a4d6-3e58110fe872 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:18.765187 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 498/1002] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jun 16 17:22:18 2022] POST /identity/v3/endpoints => generated 325 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:19.021536 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49a3bc-333e-470c-acf8-10b34943689f None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:19.022050 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49a3bc-333e-470c-acf8-10b34943689f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:19.022050 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49a3bc-333e-470c-acf8-10b34943689f None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:19.023680 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 505/1003] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:19.028754 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f413b7-1f9c-4ece-9205-ea9bfbd4eb05 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:19.029053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f413b7-1f9c-4ece-9205-ea9bfbd4eb05 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:19.029908 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f413b7-1f9c-4ece-9205-ea9bfbd4eb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:19.047415 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-a7f413b7-1f9c-4ece-9205-ea9bfbd4eb05 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:19.048625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-a7f413b7-1f9c-4ece-9205-ea9bfbd4eb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:19.123205 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 499/1004] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:19 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:19.145804 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49a3bc-333e-470c-acf8-10b34943689f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:19.256403 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49a3bc-333e-470c-acf8-10b34943689f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:19.293267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c9097-bd95-49d1-9da1-0c088469f442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:19.294079 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c9097-bd95-49d1-9da1-0c088469f442 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:19.294368 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c9097-bd95-49d1-9da1-0c088469f442 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:19.294650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c9097-bd95-49d1-9da1-0c088469f442 None admin] PATH_INFO: `/v3/services/image` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:19.295402 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c9097-bd95-49d1-9da1-0c088469f442 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:19.296925 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c9097-bd95-49d1-9da1-0c088469f442 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:19.308865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-383c9097-bd95-49d1-9da1-0c088469f442 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:22:19.308865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:19.308865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:22:19.308865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:22:19.308865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:22:19.308865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:22:19.308865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:22:19.308865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:19.308865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:22:19.308865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:19.308865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:22:19.308865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:19.308865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:19.308865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:19.308865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:19.308865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:19.308865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:19.310084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:19.310084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:19.310084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:19.310084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:19.310084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:19.310084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:19.310084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:22:19.310084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:22:19.310084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:22:19.310084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:22:19.310084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:19.310084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:19.310084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:19.310084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:19.310084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:19.310084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:19.315881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:19.315881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:19.315881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:19.315881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:19.315881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:19.315881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:19.315881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:19.315881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:19.315881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:19.315881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:19.315881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:22:19.315881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:19.315881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:22:19.315881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:19.315881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 506/1005] 10.222.0.40 () {56 vars in 1105 bytes} [Thu Jun 16 17:22:19 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:19.322343 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f413b7-1f9c-4ece-9205-ea9bfbd4eb05 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:19.366365 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:19.434833 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f413b7-1f9c-4ece-9205-ea9bfbd4eb05 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:19.473308 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa166174-de3c-468f-a6a3-c895759c0c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:19.473986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa166174-de3c-468f-a6a3-c895759c0c0e None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:19.474279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa166174-de3c-468f-a6a3-c895759c0c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:19.474565 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa166174-de3c-468f-a6a3-c895759c0c0e None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:19.475412 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa166174-de3c-468f-a6a3-c895759c0c0e None admin] RBAC: Adding query filter params (name=image) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:19.475788 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa166174-de3c-468f-a6a3-c895759c0c0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:19.477448 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa166174-de3c-468f-a6a3-c895759c0c0e None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:19.482683 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 500/1006] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:19.488655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c9097-bd95-49d1-9da1-0c088469f442 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:19.535241 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:19.597092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c9097-bd95-49d1-9da1-0c088469f442 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:19.630807 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ca888-9d71-4df7-ab51-aed616ce280b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:19.631480 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ca888-9d71-4df7-ab51-aed616ce280b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:19.631768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ca888-9d71-4df7-ab51-aed616ce280b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:19.632047 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ca888-9d71-4df7-ab51-aed616ce280b None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:19.633024 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ca888-9d71-4df7-ab51-aed616ce280b None admin] RBAC: Adding query filter params (type=image) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:19.633362 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ca888-9d71-4df7-ab51-aed616ce280b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:19.634735 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ca888-9d71-4df7-ab51-aed616ce280b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:19.640321 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 507/1007] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:19 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:20.269105 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c84c07-04cd-482f-bf47-a746a7a4150c None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.269637 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c84c07-04cd-482f-bf47-a746a7a4150c None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.269745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c84c07-04cd-482f-bf47-a746a7a4150c None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.273192 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 501/1008] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:20.276351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2351e-2989-46ab-8f2d-5c565f4ce701 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.276769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2351e-2989-46ab-8f2d-5c565f4ce701 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.276957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2351e-2989-46ab-8f2d-5c565f4ce701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.294278 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4bf2351e-2989-46ab-8f2d-5c565f4ce701 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:20.296046 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4bf2351e-2989-46ab-8f2d-5c565f4ce701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:20.367443 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 508/1009] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:20.391160 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c84c07-04cd-482f-bf47-a746a7a4150c None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.503578 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c84c07-04cd-482f-bf47-a746a7a4150c None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.505041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:20.539529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8baaa4c-9f24-442a-ab12-ac9073c87372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.540333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baaa4c-9f24-442a-ab12-ac9073c87372 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.540623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baaa4c-9f24-442a-ab12-ac9073c87372 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.540905 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baaa4c-9f24-442a-ab12-ac9073c87372 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.541605 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8baaa4c-9f24-442a-ab12-ac9073c87372 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.542973 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8baaa4c-9f24-442a-ab12-ac9073c87372 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.555298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-b8baaa4c-9f24-442a-ab12-ac9073c87372 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:22:20.555298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:20.555298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:22:20.555298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:22:20.555298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:22:20.555298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:22:20.555298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:22:20.555298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:20.555298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:22:20.555298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:20.555298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:22:20.555298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:20.555298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:20.555298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:20.555298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:20.555298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:20.555298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:20.556333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:20.556333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:20.556333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:20.556333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:20.556333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:20.556333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:20.556333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:22:20.556333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:22:20.556333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:22:20.556333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:22:20.556333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:20.556333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:20.556333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:20.556333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:20.556333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:20.556333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:20.562242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:20.562242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:20.562242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:20.562242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:20.562242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:20.562242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:20.562242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:20.562242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:20.562242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:20.562242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:20.562242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:22:20.562242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:20.562242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:22:20.562242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:20.562242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 502/1010] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:22:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:20.564699 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2351e-2989-46ab-8f2d-5c565f4ce701 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.597284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:20.670509 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2351e-2989-46ab-8f2d-5c565f4ce701 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.706230 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e2641-3fd8-4116-b120-a5a7df480590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.706908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e2641-3fd8-4116-b120-a5a7df480590 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.707196 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e2641-3fd8-4116-b120-a5a7df480590 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.707478 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e2641-3fd8-4116-b120-a5a7df480590 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.708201 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e2641-3fd8-4116-b120-a5a7df480590 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:20.708510 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e2641-3fd8-4116-b120-a5a7df480590 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.709902 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e2641-3fd8-4116-b120-a5a7df480590 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.720799 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 509/1011] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 17:22:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:20.722585 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8baaa4c-9f24-442a-ab12-ac9073c87372 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.832529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8baaa4c-9f24-442a-ab12-ac9073c87372 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.868456 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd72ae-be03-4ed2-95a3-4b8833c1a258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.869202 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd72ae-be03-4ed2-95a3-4b8833c1a258 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.869500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd72ae-be03-4ed2-95a3-4b8833c1a258 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.869838 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd72ae-be03-4ed2-95a3-4b8833c1a258 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.870727 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd72ae-be03-4ed2-95a3-4b8833c1a258 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:20.871047 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd72ae-be03-4ed2-95a3-4b8833c1a258 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.872423 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd72ae-be03-4ed2-95a3-4b8833c1a258 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.881545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 503/1012] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 17:22:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:20.883647 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e2641-3fd8-4116-b120-a5a7df480590 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.988914 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:20.993904 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ad7f7-b236-4d4a-9b48-ab2b8a1570b2 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.994450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ad7f7-b236-4d4a-9b48-ab2b8a1570b2 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.994808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ad7f7-b236-4d4a-9b48-ab2b8a1570b2 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.996259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 504/1013] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:20.997204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e2641-3fd8-4116-b120-a5a7df480590 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.001322 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a78710-6401-435a-9ab6-89dfafde6112 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.001620 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a78710-6401-435a-9ab6-89dfafde6112 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.001909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a78710-6401-435a-9ab6-89dfafde6112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.019451 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-65a78710-6401-435a-9ab6-89dfafde6112 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:21.020648 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-65a78710-6401-435a-9ab6-89dfafde6112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:21.037043 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea758f7f-c4f7-43f1-b7f8-b3df9d949456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.037734 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea758f7f-c4f7-43f1-b7f8-b3df9d949456 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.038026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea758f7f-c4f7-43f1-b7f8-b3df9d949456 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.038308 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea758f7f-c4f7-43f1-b7f8-b3df9d949456 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.039064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea758f7f-c4f7-43f1-b7f8-b3df9d949456 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.040422 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea758f7f-c4f7-43f1-b7f8-b3df9d949456 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.046140 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:21.059883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 510/1014] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/endpoints => generated 334 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:21.114823 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 505/1015] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:21 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:22:21.137520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea758f7f-c4f7-43f1-b7f8-b3df9d949456 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.282650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea758f7f-c4f7-43f1-b7f8-b3df9d949456 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.319318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9d2f5-a7c4-4440-b500-14da8598a97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.320131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9d2f5-a7c4-4440-b500-14da8598a97f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.320420 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9d2f5-a7c4-4440-b500-14da8598a97f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.320707 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9d2f5-a7c4-4440-b500-14da8598a97f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.321907 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d9d2f5-a7c4-4440-b500-14da8598a97f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.323334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d9d2f5-a7c4-4440-b500-14da8598a97f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.330542 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 511/1016] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:21.331966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a78710-6401-435a-9ab6-89dfafde6112 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.438328 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a78710-6401-435a-9ab6-89dfafde6112 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.472602 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccedaea6-dae5-4997-b9ee-f129081b806c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.473306 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccedaea6-dae5-4997-b9ee-f129081b806c None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.473601 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccedaea6-dae5-4997-b9ee-f129081b806c None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.473887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccedaea6-dae5-4997-b9ee-f129081b806c None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.474623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccedaea6-dae5-4997-b9ee-f129081b806c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.476081 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccedaea6-dae5-4997-b9ee-f129081b806c None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.503369 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 506/1017] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 17:22:21 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:21.793431 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8702e8d-eb9a-443c-9e24-4f40fff6af0e None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.793829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8702e8d-eb9a-443c-9e24-4f40fff6af0e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.794026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8702e8d-eb9a-443c-9e24-4f40fff6af0e None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.796053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 512/1018] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:21.801061 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e87ba-325e-417f-83b5-ac9d22356c39 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.801397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e87ba-325e-417f-83b5-ac9d22356c39 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.801680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e87ba-325e-417f-83b5-ac9d22356c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.819021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-4c2e87ba-325e-417f-83b5-ac9d22356c39 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:21.820200 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4c2e87ba-325e-417f-83b5-ac9d22356c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:21.822212 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:21.892239 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 507/1019] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:21 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:21.914916 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8702e8d-eb9a-443c-9e24-4f40fff6af0e None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.937894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:22.023623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8702e8d-eb9a-443c-9e24-4f40fff6af0e None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:22.057895 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2ab695-3458-44e8-a654-2391ff9d0ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:22.058477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ab695-3458-44e8-a654-2391ff9d0ec5 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:22.058763 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ab695-3458-44e8-a654-2391ff9d0ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:22.059041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ab695-3458-44e8-a654-2391ff9d0ec5 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:22.059778 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ab695-3458-44e8-a654-2391ff9d0ec5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:22.061715 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ab695-3458-44e8-a654-2391ff9d0ec5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:22.076006 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 513/1020] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 17:22:21 2022] POST /identity/v3/services => generated 242 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:23.360338 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8bbd1-b6ec-458a-94e4-b026a8bbb1b4 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.360913 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8bbd1-b6ec-458a-94e4-b026a8bbb1b4 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.360913 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8bbd1-b6ec-458a-94e4-b026a8bbb1b4 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.362322 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 508/1021] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:23.367698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afd22b-0101-4711-9c20-39782c668616 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.368110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afd22b-0101-4711-9c20-39782c668616 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.368324 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afd22b-0101-4711-9c20-39782c668616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.386505 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-29afd22b-0101-4711-9c20-39782c668616 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:23.387697 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-29afd22b-0101-4711-9c20-39782c668616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:23.463580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 514/1022] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:23.486018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8bbd1-b6ec-458a-94e4-b026a8bbb1b4 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:23.593709 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8bbd1-b6ec-458a-94e4-b026a8bbb1b4 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:23.630025 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76876209-4789-4529-81f1-238c4e3939c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:23.630847 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76876209-4789-4529-81f1-238c4e3939c8 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.631135 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76876209-4789-4529-81f1-238c4e3939c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.631413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76876209-4789-4529-81f1-238c4e3939c8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.639352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:23.644451 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76876209-4789-4529-81f1-238c4e3939c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:22:23.652581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-76876209-4789-4529-81f1-238c4e3939c8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:22:23.652581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:23.652581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:23.652581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:23.652581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:23.652581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:23.652581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:23.652581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:23.652581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:23.652581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:23.652581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:23.652581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:23.652581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:22:23.652581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:22:23.652581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:22:23.652581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:22:23.653628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:22:23.653628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:22:23.653628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:22:23.653628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:22:23.653628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:23.653628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:23.653628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:23.653628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:23.653628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:23.653628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:23.653628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:23.653628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:23.653628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:23.653628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:23.653628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:23.653628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:23.654574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:23.654574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:23.654574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:23.654574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:23.654574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:22:23.654574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:22:23.654574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:22:23.654574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:22:23.654574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:22:23.654574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:22:23.654574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:22:23.654574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:23.656389 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 509/1023] 10.222.0.40 () {56 vars in 1120 bytes} [Thu Jun 16 17:22:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:23.660173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29afd22b-0101-4711-9c20-39782c668616 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:23.669210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:23.731970 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794094a-4753-4d21-9a5e-914133aed9e2 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.732358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794094a-4753-4d21-9a5e-914133aed9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.732737 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794094a-4753-4d21-9a5e-914133aed9e2 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.734296 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 510/1024] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:22:23.739159 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78525b49-dab7-4d17-baa6-c5c2d3350dbb None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.739450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78525b49-dab7-4d17-baa6-c5c2d3350dbb None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.739728 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78525b49-dab7-4d17-baa6-c5c2d3350dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.756509 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-78525b49-dab7-4d17-baa6-c5c2d3350dbb None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:23.757987 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-78525b49-dab7-4d17-baa6-c5c2d3350dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:23.775295 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29afd22b-0101-4711-9c20-39782c668616 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:23.812403 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8d1e6-2a3d-4369-b009-147fe3ff4288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:23.813079 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8d1e6-2a3d-4369-b009-147fe3ff4288 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.813392 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8d1e6-2a3d-4369-b009-147fe3ff4288 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.813671 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8d1e6-2a3d-4369-b009-147fe3ff4288 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.814419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8d1e6-2a3d-4369-b009-147fe3ff4288 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:23.814847 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8d1e6-2a3d-4369-b009-147fe3ff4288 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:23.816209 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8d1e6-2a3d-4369-b009-147fe3ff4288 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:23.825022 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 515/1025] 10.222.0.40 () {56 vars in 1115 bytes} [Thu Jun 16 17:22:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:23.829630 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 511/1026] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:22:23.834359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8d1e6-2a3d-4369-b009-147fe3ff4288 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:23.854258 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78525b49-dab7-4d17-baa6-c5c2d3350dbb None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:23.914162 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:23.953084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8d1e6-2a3d-4369-b009-147fe3ff4288 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:23.962809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78525b49-dab7-4d17-baa6-c5c2d3350dbb None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:23.988208 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fa2d1-ee45-400f-8cd3-0c8d229a10c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:23.989003 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fa2d1-ee45-400f-8cd3-0c8d229a10c9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.989368 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fa2d1-ee45-400f-8cd3-0c8d229a10c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.989646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fa2d1-ee45-400f-8cd3-0c8d229a10c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.990871 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:24.001358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c4213-bfee-485d-adff-55cf4ba65dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:24.002167 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c4213-bfee-485d-adff-55cf4ba65dfd None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:24.002454 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c4213-bfee-485d-adff-55cf4ba65dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:24.002559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75fa2d1-ee45-400f-8cd3-0c8d229a10c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:24.002733 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c4213-bfee-485d-adff-55cf4ba65dfd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:24.003914 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75fa2d1-ee45-400f-8cd3-0c8d229a10c9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:24.004018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c4213-bfee-485d-adff-55cf4ba65dfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:24.005356 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c4213-bfee-485d-adff-55cf4ba65dfd None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:24.007395 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 512/1027] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:24.012708 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c4213-bfee-485d-adff-55cf4ba65dfd None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:24.023600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-e75fa2d1-ee45-400f-8cd3-0c8d229a10c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:24.023600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:24.023600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:24.023600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:24.023600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:24.023600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:24.023600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:24.023600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:24.023600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:24.023600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:24.023600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:24.023600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:24.023600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:22:24.023600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:22:24.023600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:22:24.024559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:22:24.024559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:24.024559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:24.024559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:24.024559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:24.024559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:24.024559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:24.024559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:24.024559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:24.024559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:24.024559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:24.024559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:24.024559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:24.024559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:24.024559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:24.024559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:24.024559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:24.025543 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:22:24.025543 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:22:24.025543 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:24.025543 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:24.025543 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 516/1028] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:22:24.031478 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fa2d1-ee45-400f-8cd3-0c8d229a10c9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:24.125102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c4213-bfee-485d-adff-55cf4ba65dfd None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:24.145050 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fa2d1-ee45-400f-8cd3-0c8d229a10c9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:24.163981 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571f21d-939c-4001-bd92-3cd134ab4465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:24.164712 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571f21d-939c-4001-bd92-3cd134ab4465 None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:24.165002 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571f21d-939c-4001-bd92-3cd134ab4465 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:24.165318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571f21d-939c-4001-bd92-3cd134ab4465 None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:24.166071 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6571f21d-939c-4001-bd92-3cd134ab4465 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:24.167577 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6571f21d-939c-4001-bd92-3cd134ab4465 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:24.171152 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:24.189306 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 513/1029] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 17:22:24 2022] POST /identity/v3/users => generated 285 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:22:24.194425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d997a-d93e-4c3b-9ad9-70bcd49a99f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:24.195255 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d997a-d93e-4c3b-9ad9-70bcd49a99f0 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:24.195551 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d997a-d93e-4c3b-9ad9-70bcd49a99f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:24.195862 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d997a-d93e-4c3b-9ad9-70bcd49a99f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:24.196664 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d997a-d93e-4c3b-9ad9-70bcd49a99f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:24.196988 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d997a-d93e-4c3b-9ad9-70bcd49a99f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:24.198582 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d997a-d93e-4c3b-9ad9-70bcd49a99f0 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:24.218037 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 517/1030] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:24.219731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571f21d-939c-4001-bd92-3cd134ab4465 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:24.235802 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feee255-cc55-49fb-987b-8326ca6b9118 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:24.236309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feee255-cc55-49fb-987b-8326ca6b9118 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:24.236438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feee255-cc55-49fb-987b-8326ca6b9118 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:24.241181 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 518/1031] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:24.243424 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e720b1c-8df5-4ad1-9571-048f19042b38 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:24.252163 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e720b1c-8df5-4ad1-9571-048f19042b38 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:24.252798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e720b1c-8df5-4ad1-9571-048f19042b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:24.271626 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0e720b1c-8df5-4ad1-9571-048f19042b38 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:24.272859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0e720b1c-8df5-4ad1-9571-048f19042b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:24.288850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:24.352838 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571f21d-939c-4001-bd92-3cd134ab4465 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:24.361302 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 519/1032] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:22:24.380613 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e720b1c-8df5-4ad1-9571-048f19042b38 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:24.392091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d760517-7469-4389-a529-523dbe075758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:24.392907 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d760517-7469-4389-a529-523dbe075758 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:24.393227 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d760517-7469-4389-a529-523dbe075758 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:24.393588 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d760517-7469-4389-a529-523dbe075758 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:24.401569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d760517-7469-4389-a529-523dbe075758 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:24.403020 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d760517-7469-4389-a529-523dbe075758 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:24.419009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-4d760517-7469-4389-a529-523dbe075758 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:22:24.419009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:24.419009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:24.419009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:24.419009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:24.419009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:24.419009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:24.419009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:24.419009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:24.419009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:24.419009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:24.419009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:24.419009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:22:24.419009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:22:24.419009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:22:24.419009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:22:24.421023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:24.421023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:24.421023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:22:24.421023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:22:24.421023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:22:24.421023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:22:24.421023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:24.421023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:24.421023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:24.421023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:24.421023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:24.421023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:24.421023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:24.421023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:24.421023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:24.421023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:24.421023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:24.421023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:24.422091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:24.422091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:24.422091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:22:24.422091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:22:24.422091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:22:24.422091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:22:24.422091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:22:24.422091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:22:24.422091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:22:24.422091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:24.425304 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 514/1033] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jun 16 17:22:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:22:24.427113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d760517-7469-4389-a529-523dbe075758 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:24.502313 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e720b1c-8df5-4ad1-9571-048f19042b38 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:24.514698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:24.536890 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d760517-7469-4389-a529-523dbe075758 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:24.541565 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051176e5-6df4-4872-b9b2-29ced2d36b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:24.542373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051176e5-6df4-4872-b9b2-29ced2d36b3d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:24.542662 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051176e5-6df4-4872-b9b2-29ced2d36b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:24.542946 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051176e5-6df4-4872-b9b2-29ced2d36b3d None admin] PATH_INFO: `/v3/services/image` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:24.543614 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051176e5-6df4-4872-b9b2-29ced2d36b3d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:24.545753 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051176e5-6df4-4872-b9b2-29ced2d36b3d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:24.551633 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:24.558250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-051176e5-6df4-4872-b9b2-29ced2d36b3d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:22:24.558250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:24.558250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:22:24.558250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:22:24.558250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:22:24.558250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:22:24.558250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:22:24.558250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:24.558250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:22:24.558250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:24.558250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:22:24.558250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:24.558250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:24.558250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:24.558250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:24.558250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:24.558250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:24.559310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:24.559310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:24.559310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:24.559310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:24.559310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:24.559310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:24.559310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:22:24.559310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:22:24.559310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:22:24.559310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:22:24.559310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:24.559310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:24.559310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:24.559310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:24.559310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:24.559310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:24.560210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:24.560210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:24.560210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:24.560210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:24.560210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:24.560210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:24.560210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:24.560210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:24.560210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:24.560210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:24.560210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:22:24.560210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:24.560210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:22:24.560210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:24.560210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 520/1034] 10.222.0.40 () {56 vars in 1105 bytes} [Thu Jun 16 17:22:24 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:22:24.565838 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051176e5-6df4-4872-b9b2-29ced2d36b3d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:24.579044 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089116a9-a1ef-4424-9dfe-448fc4a1df20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:24.579994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089116a9-a1ef-4424-9dfe-448fc4a1df20 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:24.580426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089116a9-a1ef-4424-9dfe-448fc4a1df20 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:24.580851 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089116a9-a1ef-4424-9dfe-448fc4a1df20 None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:24.581871 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089116a9-a1ef-4424-9dfe-448fc4a1df20 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:24.582338 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089116a9-a1ef-4424-9dfe-448fc4a1df20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:24.583917 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089116a9-a1ef-4424-9dfe-448fc4a1df20 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:24.603109 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 515/1035] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jun 16 17:22:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:24.609790 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089116a9-a1ef-4424-9dfe-448fc4a1df20 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:24.677872 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051176e5-6df4-4872-b9b2-29ced2d36b3d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:24.716302 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089116a9-a1ef-4424-9dfe-448fc4a1df20 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:24.716667 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf409f52-ff92-42aa-8068-8c0d29f69037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:24.717396 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf409f52-ff92-42aa-8068-8c0d29f69037 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:24.717619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf409f52-ff92-42aa-8068-8c0d29f69037 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:24.717901 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf409f52-ff92-42aa-8068-8c0d29f69037 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:24.718624 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf409f52-ff92-42aa-8068-8c0d29f69037 None admin] RBAC: Adding query filter params (name=image) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:24.718930 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf409f52-ff92-42aa-8068-8c0d29f69037 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:24.720308 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf409f52-ff92-42aa-8068-8c0d29f69037 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:24.725706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 521/1036] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:24.731051 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf409f52-ff92-42aa-8068-8c0d29f69037 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:24.755305 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b90ca-0c58-4409-8042-0d05d7c68aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:24.756274 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b90ca-0c58-4409-8042-0d05d7c68aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:24.756739 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b90ca-0c58-4409-8042-0d05d7c68aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:24.757497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b90ca-0c58-4409-8042-0d05d7c68aa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:24.773607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:24.801944 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:24.833024 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b90ca-0c58-4409-8042-0d05d7c68aa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:24.834509 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b90ca-0c58-4409-8042-0d05d7c68aa2 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:24.837722 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf409f52-ff92-42aa-8068-8c0d29f69037 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:24.873654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609f0ba-bd2f-42c5-99dd-10f83c3a11f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:24.874343 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609f0ba-bd2f-42c5-99dd-10f83c3a11f5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:24.874674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609f0ba-bd2f-42c5-99dd-10f83c3a11f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:24.874950 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609f0ba-bd2f-42c5-99dd-10f83c3a11f5 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:24.875682 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609f0ba-bd2f-42c5-99dd-10f83c3a11f5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:24.875968 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609f0ba-bd2f-42c5-99dd-10f83c3a11f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:24.877408 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609f0ba-bd2f-42c5-99dd-10f83c3a11f5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:24.882785 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 522/1037] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:24.888380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609f0ba-bd2f-42c5-99dd-10f83c3a11f5 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:24.948829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 516/1038] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:22:24 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 17:22:24.949816 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b90ca-0c58-4409-8042-0d05d7c68aa2 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:24.994983 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609f0ba-bd2f-42c5-99dd-10f83c3a11f5 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:24.996485 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:25.022457 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:25.032268 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefca141-0cbd-40f8-a397-abf7b8571124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:25.033032 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefca141-0cbd-40f8-a397-abf7b8571124 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:25.033411 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefca141-0cbd-40f8-a397-abf7b8571124 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:25.033755 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefca141-0cbd-40f8-a397-abf7b8571124 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:25.034626 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefca141-0cbd-40f8-a397-abf7b8571124 None admin] RBAC: Adding query filter params (interface=public, service_id=885570d108a842bba9364d564819a717, region_id=RegionOne) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:25.035028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefca141-0cbd-40f8-a397-abf7b8571124 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:25.036456 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefca141-0cbd-40f8-a397-abf7b8571124 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:25.041942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 523/1039] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jun 16 17:22:24 2022] GET /identity/v3/endpoints?service_id=885570d108a842bba9364d564819a717&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:25.048173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefca141-0cbd-40f8-a397-abf7b8571124 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:25.057227 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b90ca-0c58-4409-8042-0d05d7c68aa2 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:25.093214 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471d90d-c14c-41f0-834e-045d73470d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:25.094018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471d90d-c14c-41f0-834e-045d73470d1d None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:25.094313 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471d90d-c14c-41f0-834e-045d73470d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:25.094593 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471d90d-c14c-41f0-834e-045d73470d1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:25.105940 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471d90d-c14c-41f0-834e-045d73470d1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:25.109299 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471d90d-c14c-41f0-834e-045d73470d1d None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:25.129491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-1471d90d-c14c-41f0-834e-045d73470d1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:25.129491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:25.129491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:25.129491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:25.129491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:25.129491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:25.129491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:25.129491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:25.129491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:25.129491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:25.129491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:25.129491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:25.129491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:22:25.129491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:22:25.129491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:22:25.130636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:22:25.130636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:25.130636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:25.130636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:25.130636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:25.130636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:25.130636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:25.130636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:25.130636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:25.130636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:25.130636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:25.130636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:25.130636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:25.130636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:25.130636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:25.130636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:25.130636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:25.131706 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:22:25.131706 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:22:25.131706 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:25.131706 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:25.131706 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 517/1040] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:22:25.136633 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471d90d-c14c-41f0-834e-045d73470d1d None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:25.160053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefca141-0cbd-40f8-a397-abf7b8571124 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:25.195999 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb4abc-a43b-4f7b-809c-758e0e74efd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:25.196676 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb4abc-a43b-4f7b-809c-758e0e74efd5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:25.196973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb4abc-a43b-4f7b-809c-758e0e74efd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:25.197290 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb4abc-a43b-4f7b-809c-758e0e74efd5 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:25.197988 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb4abc-a43b-4f7b-809c-758e0e74efd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:25.199457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb4abc-a43b-4f7b-809c-758e0e74efd5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:25.206634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 524/1041] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jun 16 17:22:25 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:22:25.255974 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471d90d-c14c-41f0-834e-045d73470d1d None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:25.285069 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:25.298767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bdf2c-9dc3-4a99-88bd-1ca040c20e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:25.299470 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bdf2c-9dc3-4a99-88bd-1ca040c20e26 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:25.299763 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bdf2c-9dc3-4a99-88bd-1ca040c20e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:25.300197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bdf2c-9dc3-4a99-88bd-1ca040c20e26 None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:25.301082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bdf2c-9dc3-4a99-88bd-1ca040c20e26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:25.301434 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bdf2c-9dc3-4a99-88bd-1ca040c20e26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:25.302864 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bdf2c-9dc3-4a99-88bd-1ca040c20e26 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:25.320469 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 518/1042] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:25.322195 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb4abc-a43b-4f7b-809c-758e0e74efd5 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:25.359285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:25.437192 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb4abc-a43b-4f7b-809c-758e0e74efd5 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:25.473927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ec8ba-2564-42da-8f8e-85f9384c3284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:25.474646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ec8ba-2564-42da-8f8e-85f9384c3284 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:25.474930 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ec8ba-2564-42da-8f8e-85f9384c3284 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:25.475206 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ec8ba-2564-42da-8f8e-85f9384c3284 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:25.481938 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ec8ba-2564-42da-8f8e-85f9384c3284 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:25.483802 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ec8ba-2564-42da-8f8e-85f9384c3284 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:25.497525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-bc7ec8ba-2564-42da-8f8e-85f9384c3284 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:22:25.497525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:25.497525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:25.497525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:25.497525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:25.497525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:25.497525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:25.497525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:25.497525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:25.497525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:25.497525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:25.497525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:25.497525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:22:25.497525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:22:25.497525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:22:25.498695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:22:25.498695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:25.498695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:25.498695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:25.498695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:25.498695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:25.498695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:25.498695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:25.498695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:25.498695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:25.498695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:25.498695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:25.498695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:25.498695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:25.498695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:25.498695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:25.498695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:25.499686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:22:25.499686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:22:25.499686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:22:25.499686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:22:25.499686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:22:25.499686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:22:25.499686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:22:25.499686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:25.500677 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 525/1043] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:22:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:22:25.504942 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bdf2c-9dc3-4a99-88bd-1ca040c20e26 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:25.610608 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bdf2c-9dc3-4a99-88bd-1ca040c20e26 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:25.644589 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83371d28-8efe-4de4-9af2-e69e3b657892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:25.645276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83371d28-8efe-4de4-9af2-e69e3b657892 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:25.645567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83371d28-8efe-4de4-9af2-e69e3b657892 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:25.645849 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83371d28-8efe-4de4-9af2-e69e3b657892 None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:25.646636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83371d28-8efe-4de4-9af2-e69e3b657892 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:25.646951 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83371d28-8efe-4de4-9af2-e69e3b657892 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:25.648380 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83371d28-8efe-4de4-9af2-e69e3b657892 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:25.665367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 519/1044] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:22:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:25.666964 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ec8ba-2564-42da-8f8e-85f9384c3284 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:25.762012 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:25.772156 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ec8ba-2564-42da-8f8e-85f9384c3284 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:25.806905 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe94cf2-cc85-415b-8cb3-82f1b6f2eec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:25.807670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe94cf2-cc85-415b-8cb3-82f1b6f2eec4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:25.808031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe94cf2-cc85-415b-8cb3-82f1b6f2eec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:25.808403 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe94cf2-cc85-415b-8cb3-82f1b6f2eec4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:25.809354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe94cf2-cc85-415b-8cb3-82f1b6f2eec4 None admin] RBAC: Adding query filter params (role.id=3a41c48a11174f87a500cc1d6618ee65, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=689a31e26c6b454a91c3fd79678b4d07) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:25.809748 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe94cf2-cc85-415b-8cb3-82f1b6f2eec4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:25.812243 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe94cf2-cc85-415b-8cb3-82f1b6f2eec4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:25.818403 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 526/1045] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:22:25 2022] GET /identity/v3/role_assignments?user.id=689a31e26c6b454a91c3fd79678b4d07&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=3a41c48a11174f87a500cc1d6618ee65 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:26.458670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41505ffd-9cd3-40e5-a3ec-0e7184a335cf None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:26.458988 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41505ffd-9cd3-40e5-a3ec-0e7184a335cf None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:26.459254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41505ffd-9cd3-40e5-a3ec-0e7184a335cf None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:26.460659 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 520/1046] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:26.465734 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a633fa9-7282-4293-b07a-81af4be383b2 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:26.466170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a633fa9-7282-4293-b07a-81af4be383b2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:26.466343 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a633fa9-7282-4293-b07a-81af4be383b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:26.484605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-5a633fa9-7282-4293-b07a-81af4be383b2 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:26.485841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5a633fa9-7282-4293-b07a-81af4be383b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:26.553782 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 527/1047] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:26.578361 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41505ffd-9cd3-40e5-a3ec-0e7184a335cf None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:26.626717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:26.694203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41505ffd-9cd3-40e5-a3ec-0e7184a335cf None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:26.729293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ede70-5595-47e2-9ce0-542ac6c11193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:26.730087 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ede70-5595-47e2-9ce0-542ac6c11193 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:26.730375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ede70-5595-47e2-9ce0-542ac6c11193 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:26.730653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ede70-5595-47e2-9ce0-542ac6c11193 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:26.740673 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ede70-5595-47e2-9ce0-542ac6c11193 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:22:26.748811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-273ede70-5595-47e2-9ce0-542ac6c11193 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:22:26.748811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:26.748811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:26.748811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:26.748811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:26.748811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:26.748811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:26.748811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:26.748811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:26.748811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:26.748811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:26.748811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:26.748811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:22:26.748811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:22:26.748811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:22:26.748811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:22:26.749983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:22:26.749983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:22:26.749983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:22:26.749983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:22:26.749983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:26.749983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:26.749983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:26.749983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:26.749983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:26.749983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:26.749983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:26.749983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:26.749983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:26.749983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:26.749983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:26.749983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:26.750889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:26.750889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:26.750889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:26.750889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:26.750889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:22:26.750889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:22:26.750889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:22:26.750889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:22:26.750889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:22:26.750889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:22:26.750889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:22:26.750889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:26.752384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 521/1048] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:22:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:26.756186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a633fa9-7282-4293-b07a-81af4be383b2 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:26.851025 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:26.869241 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a633fa9-7282-4293-b07a-81af4be383b2 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:26.907544 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ab83f-3d11-418e-a2d7-d02a3e336084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:26.908246 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ab83f-3d11-418e-a2d7-d02a3e336084 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:26.908534 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ab83f-3d11-418e-a2d7-d02a3e336084 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:26.908817 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ab83f-3d11-418e-a2d7-d02a3e336084 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:26.909616 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ab83f-3d11-418e-a2d7-d02a3e336084 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:26.909932 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ab83f-3d11-418e-a2d7-d02a3e336084 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:26.911361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ab83f-3d11-418e-a2d7-d02a3e336084 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:26.921282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 528/1049] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:22:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:26.926784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ede70-5595-47e2-9ce0-542ac6c11193 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:27.043116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ede70-5595-47e2-9ce0-542ac6c11193 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:27.044760 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:27.089800 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a7a61-d3cf-4b01-ba4f-21997eb12890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:27.090699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a7a61-d3cf-4b01-ba4f-21997eb12890 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:27.090989 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a7a61-d3cf-4b01-ba4f-21997eb12890 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:27.091268 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a7a61-d3cf-4b01-ba4f-21997eb12890 None admin] PATH_INFO: `/v3/users/439ce3ee63a54e8fafcd10d9872821be` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:27.110144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a7a61-d3cf-4b01-ba4f-21997eb12890 None admin] RBAC: Authorizing `identity:get_user(user_id=439ce3ee63a54e8fafcd10d9872821be)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:27.112045 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a7a61-d3cf-4b01-ba4f-21997eb12890 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:27.115751 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 522/1050] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:22:26 2022] GET /identity/v3/users/439ce3ee63a54e8fafcd10d9872821be => generated 285 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:27.123139 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ab83f-3d11-418e-a2d7-d02a3e336084 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:27.251109 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ab83f-3d11-418e-a2d7-d02a3e336084 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:27.302995 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea00a51-3b86-4d65-a842-161c90479593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:27.303667 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea00a51-3b86-4d65-a842-161c90479593 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:27.303961 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea00a51-3b86-4d65-a842-161c90479593 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:27.304293 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea00a51-3b86-4d65-a842-161c90479593 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:27.322012 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:27.386907 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea00a51-3b86-4d65-a842-161c90479593 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:27.388365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea00a51-3b86-4d65-a842-161c90479593 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:27.421899 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5bf5e-deb5-4ed2-b5c1-656dd8dded94 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:27.422201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5bf5e-deb5-4ed2-b5c1-656dd8dded94 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:27.422584 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5bf5e-deb5-4ed2-b5c1-656dd8dded94 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:27.423871 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 523/1051] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:27.429367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3bb5d-34ef-4072-a8d0-4593168521b1 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:27.429666 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3bb5d-34ef-4072-a8d0-4593168521b1 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:27.429956 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3bb5d-34ef-4072-a8d0-4593168521b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:27.448663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-4fb3bb5d-34ef-4072-a8d0-4593168521b1 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:27.449910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4fb3bb5d-34ef-4072-a8d0-4593168521b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:27.503969 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 529/1052] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:22:27 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:22:27.509695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea00a51-3b86-4d65-a842-161c90479593 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:27.545773 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 524/1053] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:22:27.563949 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:27.569869 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3bb5d-34ef-4072-a8d0-4593168521b1 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:27.629855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea00a51-3b86-4d65-a842-161c90479593 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:27.675437 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91a5a4-221c-4d5d-a83a-0854fa472fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:27.676305 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91a5a4-221c-4d5d-a83a-0854fa472fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:27.676597 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91a5a4-221c-4d5d-a83a-0854fa472fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:27.676880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91a5a4-221c-4d5d-a83a-0854fa472fc5 None admin] PATH_INFO: `/v3/projects/bdd732d58dff46eab9a48d755dbccc92` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:27.678482 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e91a5a4-221c-4d5d-a83a-0854fa472fc5 None admin] RBAC: Authorizing `identity:get_project(project_id=bdd732d58dff46eab9a48d755dbccc92)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:27.680015 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e91a5a4-221c-4d5d-a83a-0854fa472fc5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:27.681853 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:27.682311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 530/1054] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:22:27 2022] GET /identity/v3/projects/bdd732d58dff46eab9a48d755dbccc92 => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:27.689775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91a5a4-221c-4d5d-a83a-0854fa472fc5 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:27.695428 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3bb5d-34ef-4072-a8d0-4593168521b1 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:27.737452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc3aad-25e1-4b6c-84e5-051f05090424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:27.738259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc3aad-25e1-4b6c-84e5-051f05090424 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:27.738808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc3aad-25e1-4b6c-84e5-051f05090424 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:27.738808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc3aad-25e1-4b6c-84e5-051f05090424 None admin] PATH_INFO: `/v3/services/image` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:27.739484 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc3aad-25e1-4b6c-84e5-051f05090424 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:27.740862 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc3aad-25e1-4b6c-84e5-051f05090424 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:27.753761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-e9bc3aad-25e1-4b6c-84e5-051f05090424 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:22:27.753761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:27.753761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:22:27.753761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:22:27.753761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:22:27.753761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:22:27.753761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:22:27.753761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:27.753761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:22:27.753761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:27.753761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:22:27.753761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:27.753761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:27.753761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:27.753761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:27.753761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:27.753761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:27.754858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:27.754858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:27.754858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:27.754858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:27.754858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:27.754858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:27.754858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:22:27.754858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:22:27.754858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:22:27.754858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:22:27.754858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:27.754858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:27.754858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:27.754858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:27.754858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:27.754858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:27.755881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:27.755881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:27.755881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:27.755881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:27.755881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:27.755881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:27.755881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:27.755881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:27.755881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:27.755881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:27.755881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:22:27.755881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:22:27.755881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:22:27.755881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:27.755881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 525/1055] 10.222.0.40 () {56 vars in 1105 bytes} [Thu Jun 16 17:22:27 2022] GET /identity/v3/services/image => generated 86 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:22:27.761271 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc3aad-25e1-4b6c-84e5-051f05090424 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:27.823016 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91a5a4-221c-4d5d-a83a-0854fa472fc5 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:27.838885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:27.861502 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac225aa0-277f-4d1c-b467-2250a9529264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:27.862175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac225aa0-277f-4d1c-b467-2250a9529264 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:27.862465 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac225aa0-277f-4d1c-b467-2250a9529264 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:27.862744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac225aa0-277f-4d1c-b467-2250a9529264 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:27.863586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac225aa0-277f-4d1c-b467-2250a9529264 None admin] RBAC: Adding query filter params (role.id=25a16d8ea4084e9eacdebb7a4dcd2337, scope.project.id=bdd732d58dff46eab9a48d755dbccc92, user.id=439ce3ee63a54e8fafcd10d9872821be) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:27.863912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac225aa0-277f-4d1c-b467-2250a9529264 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:27.865373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac225aa0-277f-4d1c-b467-2250a9529264 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:27.871380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 531/1056] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:22:27 2022] GET /identity/v3/role_assignments?user.id=439ce3ee63a54e8fafcd10d9872821be&scope.project.id=bdd732d58dff46eab9a48d755dbccc92&role.id=25a16d8ea4084e9eacdebb7a4dcd2337 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:27.877858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc3aad-25e1-4b6c-84e5-051f05090424 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:27.918180 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dab71c-767b-48b5-9351-bf3d53644b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:27.918878 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dab71c-767b-48b5-9351-bf3d53644b49 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:27.919168 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dab71c-767b-48b5-9351-bf3d53644b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:27.919486 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dab71c-767b-48b5-9351-bf3d53644b49 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:27.920245 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dab71c-767b-48b5-9351-bf3d53644b49 None admin] RBAC: Adding query filter params (name=image) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:27.920564 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dab71c-767b-48b5-9351-bf3d53644b49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:27.922029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dab71c-767b-48b5-9351-bf3d53644b49 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:27.932905 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 526/1057] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:27.934680 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac225aa0-277f-4d1c-b467-2250a9529264 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:28.062473 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac225aa0-277f-4d1c-b467-2250a9529264 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:28.096990 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb9f6c-40ca-43f8-9fa4-d4e02bd30275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:28.097703 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb9f6c-40ca-43f8-9fa4-d4e02bd30275 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:28.097991 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb9f6c-40ca-43f8-9fa4-d4e02bd30275 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:28.098269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb9f6c-40ca-43f8-9fa4-d4e02bd30275 None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:28.098986 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb9f6c-40ca-43f8-9fa4-d4e02bd30275 None admin] RBAC: Adding query filter params (type=image) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:28.099294 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb9f6c-40ca-43f8-9fa4-d4e02bd30275 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:28.100670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb9f6c-40ca-43f8-9fa4-d4e02bd30275 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:28.106263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 532/1058] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:28.112184 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dab71c-767b-48b5-9351-bf3d53644b49 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:28.135101 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:28.228246 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dab71c-767b-48b5-9351-bf3d53644b49 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:28.270890 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c27f9f-0c15-40fb-a782-c954514666e0 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:28.271210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c27f9f-0c15-40fb-a782-c954514666e0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:28.271456 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c27f9f-0c15-40fb-a782-c954514666e0 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:28.272970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 533/1059] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:22:28.273077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e2d7b-401b-4410-b118-6293c5a4c88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:28.273700 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e2d7b-401b-4410-b118-6293c5a4c88d None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:28.274051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e2d7b-401b-4410-b118-6293c5a4c88d None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:28.274342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e2d7b-401b-4410-b118-6293c5a4c88d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:28.275088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e2d7b-401b-4410-b118-6293c5a4c88d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:28.276655 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e2d7b-401b-4410-b118-6293c5a4c88d None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:28.277662 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81e792-95d4-4752-a0e5-a1dfae88b75a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:28.277955 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81e792-95d4-4752-a0e5-a1dfae88b75a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:28.278238 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81e792-95d4-4752-a0e5-a1dfae88b75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:28.295748 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-7c81e792-95d4-4752-a0e5-a1dfae88b75a None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:28.296927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7c81e792-95d4-4752-a0e5-a1dfae88b75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:28.305233 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 527/1060] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jun 16 17:22:28 2022] POST /identity/v3/endpoints => generated 327 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:28.339883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:28.388233 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 534/1061] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:22:28.410906 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e2d7b-401b-4410-b118-6293c5a4c88d None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:28.493715 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:28.526496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e2d7b-401b-4410-b118-6293c5a4c88d None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:28.563611 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7db945-f4da-4334-a815-d2dec6ad7abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:28.564443 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7db945-f4da-4334-a815-d2dec6ad7abf None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:28.564735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7db945-f4da-4334-a815-d2dec6ad7abf None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:28.565024 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7db945-f4da-4334-a815-d2dec6ad7abf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:28.575351 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7db945-f4da-4334-a815-d2dec6ad7abf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:22:28.583755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-5e7db945-f4da-4334-a815-d2dec6ad7abf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:22:28.583755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:28.583755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:28.583755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:28.583755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:28.583755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:28.583755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:28.583755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:28.583755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:28.583755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:28.583755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:28.583755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:28.583755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:22:28.583755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:22:28.583755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:22:28.583755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:22:28.584753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:22:28.584753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:22:28.584753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:22:28.584753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:22:28.584753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:28.584753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:28.584753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:28.584753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:28.584753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:28.584753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:28.584753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:28.584753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:28.584753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:28.584753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:28.584753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:28.584753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:28.586155 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:28.586155 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:28.586155 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:28.586155 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:28.586155 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:22:28.586155 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:22:28.586155 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:22:28.586155 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:22:28.586155 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:22:28.586155 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:22:28.586155 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:22:28.586155 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:28.586155 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 528/1062] 10.222.0.40 () {56 vars in 1120 bytes} [Thu Jun 16 17:22:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:28.591159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81e792-95d4-4752-a0e5-a1dfae88b75a None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:28.693725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81e792-95d4-4752-a0e5-a1dfae88b75a None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:28.728075 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc9aec-f9f9-4323-8223-cd8d899db545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:28.728748 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc9aec-f9f9-4323-8223-cd8d899db545 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:28.729032 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc9aec-f9f9-4323-8223-cd8d899db545 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:28.729330 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc9aec-f9f9-4323-8223-cd8d899db545 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:28.730076 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc9aec-f9f9-4323-8223-cd8d899db545 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:28.730382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc9aec-f9f9-4323-8223-cd8d899db545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:28.731736 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc9aec-f9f9-4323-8223-cd8d899db545 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:28.745182 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 535/1063] 10.222.0.40 () {56 vars in 1115 bytes} [Thu Jun 16 17:22:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:28.746449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7db945-f4da-4334-a815-d2dec6ad7abf None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:28.851205 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7db945-f4da-4334-a815-d2dec6ad7abf None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:28.885640 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae851f3e-bf7e-41a5-b87a-30fe92df2393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:28.886500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae851f3e-bf7e-41a5-b87a-30fe92df2393 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:28.886794 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae851f3e-bf7e-41a5-b87a-30fe92df2393 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:28.887072 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae851f3e-bf7e-41a5-b87a-30fe92df2393 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:28.889183 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:28.899847 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae851f3e-bf7e-41a5-b87a-30fe92df2393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:28.901589 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae851f3e-bf7e-41a5-b87a-30fe92df2393 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:28.919058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-ae851f3e-bf7e-41a5-b87a-30fe92df2393 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:28.919058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:28.919058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:28.919058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:28.919058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:28.919058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:28.919058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:28.919058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:28.919058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:28.919058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:28.919058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:28.919058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:28.919058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:22:28.919058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:22:28.919058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:22:28.921010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:22:28.921010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:28.921010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:28.921010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:28.921010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:28.921010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:28.921010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:28.921010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:28.921010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:28.921010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:28.921010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:28.921010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:28.921010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:28.921010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:28.921010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:28.921010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:28.921010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:28.922003 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:22:28.922003 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:22:28.922003 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:28.922003 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:28.922779 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 529/1064] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:28.926472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc9aec-f9f9-4323-8223-cd8d899db545 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:28.958796 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:29.035581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc9aec-f9f9-4323-8223-cd8d899db545 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:29.070708 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a8588-ed37-406b-86c5-a7222c5e9cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:29.071380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a8588-ed37-406b-86c5-a7222c5e9cdd None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:29.071666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a8588-ed37-406b-86c5-a7222c5e9cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:29.071942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a8588-ed37-406b-86c5-a7222c5e9cdd None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:29.072667 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a8588-ed37-406b-86c5-a7222c5e9cdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:29.072976 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a8588-ed37-406b-86c5-a7222c5e9cdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:29.074450 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a8588-ed37-406b-86c5-a7222c5e9cdd None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:29.093686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 536/1065] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:29.095332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae851f3e-bf7e-41a5-b87a-30fe92df2393 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:29.206173 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae851f3e-bf7e-41a5-b87a-30fe92df2393 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:29.252425 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c30fd0-60f3-42f5-8618-c42c786fff30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:29.253261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c30fd0-60f3-42f5-8618-c42c786fff30 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:29.253557 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c30fd0-60f3-42f5-8618-c42c786fff30 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:29.253842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c30fd0-60f3-42f5-8618-c42c786fff30 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:29.261712 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c30fd0-60f3-42f5-8618-c42c786fff30 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:29.263785 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c30fd0-60f3-42f5-8618-c42c786fff30 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:29.278208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-89c30fd0-60f3-42f5-8618-c42c786fff30 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:22:29.278208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:29.278208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:29.278208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:29.278208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:29.278208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:29.278208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:29.278208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:29.278208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:29.278208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:29.278208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:29.278208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:29.278208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:22:29.278208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:22:29.278208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:22:29.278208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:22:29.279784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:29.279784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:29.279784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:22:29.279784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:22:29.279784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:22:29.279784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:22:29.279784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:29.279784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:29.279784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:29.279784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:29.279784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:29.279784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:29.279784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:29.279784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:29.279784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:29.279784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:29.279784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:29.279784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:29.280801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:29.280801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:29.280801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:22:29.280801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:22:29.280801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:22:29.280801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:22:29.280801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:22:29.280801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:22:29.280801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:22:29.280801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:29.285297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 530/1066] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jun 16 17:22:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:29.286680 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a8588-ed37-406b-86c5-a7222c5e9cdd None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:29.385013 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:29.397414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a8588-ed37-406b-86c5-a7222c5e9cdd None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:29.432791 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedaa85d-5143-4cdd-976e-b1992e8ca258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:29.433530 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedaa85d-5143-4cdd-976e-b1992e8ca258 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:29.433822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedaa85d-5143-4cdd-976e-b1992e8ca258 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:29.434104 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedaa85d-5143-4cdd-976e-b1992e8ca258 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:29.435475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedaa85d-5143-4cdd-976e-b1992e8ca258 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:29.436008 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedaa85d-5143-4cdd-976e-b1992e8ca258 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:29.437755 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedaa85d-5143-4cdd-976e-b1992e8ca258 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:29.462929 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 537/1067] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jun 16 17:22:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:29.464518 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c30fd0-60f3-42f5-8618-c42c786fff30 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:29.516828 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:29.571069 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c30fd0-60f3-42f5-8618-c42c786fff30 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:29.609014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6dce44-866d-46d8-8b62-9985e1c64a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:29.609860 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6dce44-866d-46d8-8b62-9985e1c64a55 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:29.610147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6dce44-866d-46d8-8b62-9985e1c64a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:29.610424 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6dce44-866d-46d8-8b62-9985e1c64a55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:29.621630 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6dce44-866d-46d8-8b62-9985e1c64a55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:29.623988 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6dce44-866d-46d8-8b62-9985e1c64a55 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:29.641430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-cb6dce44-866d-46d8-8b62-9985e1c64a55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:29.641430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:29.641430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:29.641430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:29.641430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:29.641430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:29.641430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:29.641430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:29.641430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:29.641430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:29.641430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:29.641430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:29.641430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:22:29.641430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:22:29.641430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:22:29.642358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:22:29.642358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:29.642358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:29.642358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:29.642358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:29.642358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:29.642358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:29.642358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:29.642358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:29.642358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:29.642358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:29.642358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:29.642358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:29.642358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:29.642358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:29.642358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:29.642358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:29.643277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:22:29.643277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:22:29.643277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:29.643277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:29.645495 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 531/1068] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:29.648139 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedaa85d-5143-4cdd-976e-b1992e8ca258 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:29.746895 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedaa85d-5143-4cdd-976e-b1992e8ca258 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:29.783080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74130733-2e72-4df8-9686-2c19d245d952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:29.783751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74130733-2e72-4df8-9686-2c19d245d952 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:29.784034 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74130733-2e72-4df8-9686-2c19d245d952 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:29.784308 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74130733-2e72-4df8-9686-2c19d245d952 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:29.785027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74130733-2e72-4df8-9686-2c19d245d952 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:29.785377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74130733-2e72-4df8-9686-2c19d245d952 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:29.786725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74130733-2e72-4df8-9686-2c19d245d952 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:29.802309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 538/1069] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:22:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:29.804031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6dce44-866d-46d8-8b62-9985e1c64a55 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:29.905782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:29.910743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6dce44-866d-46d8-8b62-9985e1c64a55 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:29.944474 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40a3e1-4b9a-4de3-aaaf-0ac223bcb7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:29.945304 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40a3e1-4b9a-4de3-aaaf-0ac223bcb7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:29.945601 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40a3e1-4b9a-4de3-aaaf-0ac223bcb7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:29.945884 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40a3e1-4b9a-4de3-aaaf-0ac223bcb7f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:29.953181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf40a3e1-4b9a-4de3-aaaf-0ac223bcb7f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:29.954659 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf40a3e1-4b9a-4de3-aaaf-0ac223bcb7f7 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:29.969718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-cf40a3e1-4b9a-4de3-aaaf-0ac223bcb7f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:22:29.969718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:29.969718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:29.969718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:29.969718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:29.969718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:29.969718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:29.969718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:29.969718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:29.969718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:29.969718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:29.969718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:29.969718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:22:29.969718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:22:29.969718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:22:29.970679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:22:29.970679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:29.970679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:29.970679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:29.970679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:29.970679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:29.970679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:29.970679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:29.970679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:29.970679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:29.970679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:29.970679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:29.970679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:29.970679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:29.970679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:29.970679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:29.970679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:29.971623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:22:29.971623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:22:29.971623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:22:29.971623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:22:29.971623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:22:29.971623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:22:29.971623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:22:29.971623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:29.973481 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 532/1070] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:22:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:29.976994 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74130733-2e72-4df8-9686-2c19d245d952 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:29.990395 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:30.048706 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf06f07-e75f-45a7-bb33-af69e55facdf None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:30.049008 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf06f07-e75f-45a7-bb33-af69e55facdf None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:30.049328 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf06f07-e75f-45a7-bb33-af69e55facdf None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:30.050830 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 533/1071] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:30.055558 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860a3b8-8ff3-4ae5-9b52-cea80e95868d None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:30.055852 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860a3b8-8ff3-4ae5-9b52-cea80e95868d None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:30.056144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860a3b8-8ff3-4ae5-9b52-cea80e95868d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:30.073547 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-4860a3b8-8ff3-4ae5-9b52-cea80e95868d None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:30.074744 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4860a3b8-8ff3-4ae5-9b52-cea80e95868d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:30.090979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74130733-2e72-4df8-9686-2c19d245d952 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:30.126415 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5c46a-1b6d-4b07-8328-2f83742c7f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:30.127089 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5c46a-1b6d-4b07-8328-2f83742c7f01 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:30.127379 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5c46a-1b6d-4b07-8328-2f83742c7f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:30.127658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5c46a-1b6d-4b07-8328-2f83742c7f01 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:30.128451 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5c46a-1b6d-4b07-8328-2f83742c7f01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:30.128766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5c46a-1b6d-4b07-8328-2f83742c7f01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:30.130250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5c46a-1b6d-4b07-8328-2f83742c7f01 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:30.143805 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 534/1072] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:22:30.148747 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 539/1073] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:22:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:30.149979 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860a3b8-8ff3-4ae5-9b52-cea80e95868d None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:30.165785 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5c46a-1b6d-4b07-8328-2f83742c7f01 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:30.235937 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:30.255206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:30.272500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860a3b8-8ff3-4ae5-9b52-cea80e95868d None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:30.274778 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5c46a-1b6d-4b07-8328-2f83742c7f01 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:30.310891 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4209590-948b-402a-8907-cdae7d80ff64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:30.311566 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6b1ad-dfd6-4d69-9097-b51722809e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:30.311973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4209590-948b-402a-8907-cdae7d80ff64 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:30.312263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4209590-948b-402a-8907-cdae7d80ff64 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:30.312467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6b1ad-dfd6-4d69-9097-b51722809e63 None admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:30.312566 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4209590-948b-402a-8907-cdae7d80ff64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:30.312758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6b1ad-dfd6-4d69-9097-b51722809e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:30.313044 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6b1ad-dfd6-4d69-9097-b51722809e63 None admin] PATH_INFO: `/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/689a31e26c6b454a91c3fd79678b4d07/roles/3a41c48a11174f87a500cc1d6618ee65` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:30.315419 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6b1ad-dfd6-4d69-9097-b51722809e63 None admin] RBAC: Authorizing `identity:create_grant(project_id=35a535a305e740368b7a4d77d0ad22a0, user_id=689a31e26c6b454a91c3fd79678b4d07, role_id=3a41c48a11174f87a500cc1d6618ee65)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:30.316833 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6b1ad-dfd6-4d69-9097-b51722809e63 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:30.323186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4209590-948b-402a-8907-cdae7d80ff64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:22:30.331475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 535/1074] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:22:30 2022] PUT /identity/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/689a31e26c6b454a91c3fd79678b4d07/roles/3a41c48a11174f87a500cc1d6618ee65 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:22:30.331704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-d4209590-948b-402a-8907-cdae7d80ff64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:22:30.331704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:30.331704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:30.331704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:30.331704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:30.331704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:30.331704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:30.331704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:30.331704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:30.331704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:30.331704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:30.331704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:30.331704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:22:30.331704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:22:30.331704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:22:30.331704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:22:30.332661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:22:30.332661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:22:30.332661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:22:30.332661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:22:30.332661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:30.332661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:30.332661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:30.332661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:30.332661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:30.332661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:30.332661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:30.332661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:30.332661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:30.332661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:30.332661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:30.332661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:30.334039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:30.334039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:30.334039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:30.334039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:30.334039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:22:30.334039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:22:30.334039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:22:30.334039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:22:30.334039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:22:30.334039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:22:30.334039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:22:30.334039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:30.338237 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 540/1075] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:22:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:22:30.339399 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6b1ad-dfd6-4d69-9097-b51722809e63 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:30.453320 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6b1ad-dfd6-4d69-9097-b51722809e63 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:30.499503 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d1af0-9b18-4c33-8aba-46f2cd07abd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:30.500212 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d1af0-9b18-4c33-8aba-46f2cd07abd8 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:30.500503 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d1af0-9b18-4c33-8aba-46f2cd07abd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:30.500787 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d1af0-9b18-4c33-8aba-46f2cd07abd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:30.501627 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d1af0-9b18-4c33-8aba-46f2cd07abd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:30.501952 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d1af0-9b18-4c33-8aba-46f2cd07abd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:30.503386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d1af0-9b18-4c33-8aba-46f2cd07abd8 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:30.514157 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 536/1076] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:22:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:30.519920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4209590-948b-402a-8907-cdae7d80ff64 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:30.562518 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c9ac6-ff1e-4b57-ba2f-04a04965a6a9 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:30.562831 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c9ac6-ff1e-4b57-ba2f-04a04965a6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:30.563105 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c9ac6-ff1e-4b57-ba2f-04a04965a6a9 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:30.564617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 537/1077] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:30.570079 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b7aa9-4051-494f-975c-ceffdb0c2479 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:30.570385 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b7aa9-4051-494f-975c-ceffdb0c2479 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:30.570674 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b7aa9-4051-494f-975c-ceffdb0c2479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:30.590321 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-405b7aa9-4051-494f-975c-ceffdb0c2479 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:30.592130 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-405b7aa9-4051-494f-975c-ceffdb0c2479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:30.595753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:30.635682 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4209590-948b-402a-8907-cdae7d80ff64 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:30.663326 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:30.667860 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 538/1078] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:22:30.676213 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b3486-f4d5-48bf-ae88-e52f1537102f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:30.677335 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b3486-f4d5-48bf-ae88-e52f1537102f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:30.677812 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b3486-f4d5-48bf-ae88-e52f1537102f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:30.678243 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b3486-f4d5-48bf-ae88-e52f1537102f None admin] PATH_INFO: `/v3/users/439ce3ee63a54e8fafcd10d9872821be` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:30.679592 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b3486-f4d5-48bf-ae88-e52f1537102f None admin] RBAC: Authorizing `identity:get_user(user_id=439ce3ee63a54e8fafcd10d9872821be)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:30.681199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b3486-f4d5-48bf-ae88-e52f1537102f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:30.682986 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 541/1079] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:22:30 2022] GET /identity/v3/users/439ce3ee63a54e8fafcd10d9872821be => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:30.689256 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b7aa9-4051-494f-975c-ceffdb0c2479 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:30.693481 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b3486-f4d5-48bf-ae88-e52f1537102f None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:30.802122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b7aa9-4051-494f-975c-ceffdb0c2479 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:30.804573 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b3486-f4d5-48bf-ae88-e52f1537102f None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:30.842733 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bc132-d344-4979-b0b7-d2296755e6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:30.843416 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bc132-d344-4979-b0b7-d2296755e6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:30.843685 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bc132-d344-4979-b0b7-d2296755e6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:30.843972 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bc132-d344-4979-b0b7-d2296755e6f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:30.853607 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4ea38-daaf-4c5d-8338-d268edf6fb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:30.854489 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4ea38-daaf-4c5d-8338-d268edf6fb9b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:30.854819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4ea38-daaf-4c5d-8338-d268edf6fb9b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:30.855102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4ea38-daaf-4c5d-8338-d268edf6fb9b None admin] PATH_INFO: `/v3/projects/bdd732d58dff46eab9a48d755dbccc92` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:30.856672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4ea38-daaf-4c5d-8338-d268edf6fb9b None admin] RBAC: Authorizing `identity:get_project(project_id=bdd732d58dff46eab9a48d755dbccc92)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:30.859391 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4ea38-daaf-4c5d-8338-d268edf6fb9b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:30.867055 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 539/1080] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:22:30 2022] GET /identity/v3/projects/bdd732d58dff46eab9a48d755dbccc92 => generated 311 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:30.867779 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4ea38-daaf-4c5d-8338-d268edf6fb9b None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:30.878848 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:30.921271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:30.925793 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882bc132-d344-4979-b0b7-d2296755e6f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:30.928087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882bc132-d344-4979-b0b7-d2296755e6f4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:30.975145 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4ea38-daaf-4c5d-8338-d268edf6fb9b None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:31.012783 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b7b61-7ef6-4620-90ef-2294c98a2fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:31.013756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b7b61-7ef6-4620-90ef-2294c98a2fc4 None admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:31.014163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b7b61-7ef6-4620-90ef-2294c98a2fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:31.014909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b7b61-7ef6-4620-90ef-2294c98a2fc4 None admin] PATH_INFO: `/v3/projects/bdd732d58dff46eab9a48d755dbccc92/users/439ce3ee63a54e8fafcd10d9872821be/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:31.017325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b7b61-7ef6-4620-90ef-2294c98a2fc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=bdd732d58dff46eab9a48d755dbccc92, user_id=439ce3ee63a54e8fafcd10d9872821be, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:31.019099 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b7b61-7ef6-4620-90ef-2294c98a2fc4 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:31.034542 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 540/1081] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:22:30 2022] PUT /identity/v3/projects/bdd732d58dff46eab9a48d755dbccc92/users/439ce3ee63a54e8fafcd10d9872821be/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:31.053907 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 542/1082] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:22:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:22:31.059723 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b7b61-7ef6-4620-90ef-2294c98a2fc4 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:31.137931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:31.187287 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b7b61-7ef6-4620-90ef-2294c98a2fc4 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:31.228380 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583130a-6d35-4e37-bf56-c6a664bd3293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:31.229249 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583130a-6d35-4e37-bf56-c6a664bd3293 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:31.229547 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583130a-6d35-4e37-bf56-c6a664bd3293 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:31.229835 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583130a-6d35-4e37-bf56-c6a664bd3293 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:31.241570 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583130a-6d35-4e37-bf56-c6a664bd3293 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:31.243041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583130a-6d35-4e37-bf56-c6a664bd3293 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:31.262244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-f583130a-6d35-4e37-bf56-c6a664bd3293 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:31.262244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:31.262244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:31.262244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:31.262244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:31.262244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:31.262244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:31.262244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:31.262244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:31.262244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:31.262244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:31.262244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:31.262244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:22:31.262244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:22:31.262244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:22:31.263699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:22:31.263699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:31.263699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:31.263699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:31.263699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:31.263699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:31.263699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:31.263699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:31.263699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:31.263699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:31.263699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:31.263699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:31.263699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:31.263699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:31.263699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:31.263699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:31.263699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:31.264650 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:22:31.264650 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:22:31.264650 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:31.264650 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:31.264879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 541/1083] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:31.270798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bc132-d344-4979-b0b7-d2296755e6f4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:31.369636 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:31.377440 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bc132-d344-4979-b0b7-d2296755e6f4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:31.414300 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d316e0-e38e-41ad-a344-51002e05dc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:31.414976 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d316e0-e38e-41ad-a344-51002e05dc0b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:31.415259 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d316e0-e38e-41ad-a344-51002e05dc0b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:31.415540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d316e0-e38e-41ad-a344-51002e05dc0b None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:31.416269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d316e0-e38e-41ad-a344-51002e05dc0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:31.416742 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d316e0-e38e-41ad-a344-51002e05dc0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:31.417967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d316e0-e38e-41ad-a344-51002e05dc0b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:31.429648 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 543/1084] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:22:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:32.508648 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0856b6f-8dc2-4fed-97b2-e5928443b605 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:32.509239 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0856b6f-8dc2-4fed-97b2-e5928443b605 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:32.509239 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0856b6f-8dc2-4fed-97b2-e5928443b605 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:32.510623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 542/1085] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:32.515803 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a02e4d-d711-453f-9516-36e765f05041 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:32.516256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a02e4d-d711-453f-9516-36e765f05041 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:32.516361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a02e4d-d711-453f-9516-36e765f05041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:32.533378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-87a02e4d-d711-453f-9516-36e765f05041 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:32.534560 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-87a02e4d-d711-453f-9516-36e765f05041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:32.604803 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 544/1086] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:32.628107 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0856b6f-8dc2-4fed-97b2-e5928443b605 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:32.730354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0856b6f-8dc2-4fed-97b2-e5928443b605 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:32.750019 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:32.765140 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6b13a-2b63-4985-9855-02623bfdf226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:32.765922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6b13a-2b63-4985-9855-02623bfdf226 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:32.766214 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6b13a-2b63-4985-9855-02623bfdf226 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:32.766492 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6b13a-2b63-4985-9855-02623bfdf226 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:32.776553 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6b13a-2b63-4985-9855-02623bfdf226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:22:32.785625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-15b6b13a-2b63-4985-9855-02623bfdf226 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:22:32.785625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:32.785625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:32.785625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:32.785625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:32.785625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:32.785625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:32.785625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:32.785625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:32.785625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:32.785625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:32.785625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:32.785625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:22:32.785625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:22:32.785625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:22:32.785625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:22:32.786903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:22:32.786903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:22:32.786903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:22:32.786903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:22:32.786903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:32.786903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:32.786903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:32.786903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:32.786903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:32.786903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:32.786903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:32.786903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:32.786903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:32.786903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:32.786903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:32.786903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:32.787857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:32.787857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:32.787857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:32.787857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:32.787857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:22:32.787857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:22:32.787857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:22:32.787857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:22:32.787857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:22:32.787857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:22:32.787857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:22:32.787857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:32.787857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 543/1087] 10.222.0.40 () {56 vars in 1120 bytes} [Thu Jun 16 17:22:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:32.792796 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a02e4d-d711-453f-9516-36e765f05041 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:32.876090 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:32.891630 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a02e4d-d711-453f-9516-36e765f05041 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:32.924881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c10e6a-cf01-42ca-8376-fd77f6e94085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:32.925581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c10e6a-cf01-42ca-8376-fd77f6e94085 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:32.925864 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c10e6a-cf01-42ca-8376-fd77f6e94085 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:32.926141 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c10e6a-cf01-42ca-8376-fd77f6e94085 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:32.926891 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c10e6a-cf01-42ca-8376-fd77f6e94085 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:32.927200 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c10e6a-cf01-42ca-8376-fd77f6e94085 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:32.928561 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c10e6a-cf01-42ca-8376-fd77f6e94085 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:32.936786 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 545/1088] 10.222.0.40 () {56 vars in 1115 bytes} [Thu Jun 16 17:22:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:32.942230 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6b13a-2b63-4985-9855-02623bfdf226 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:33.045051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6b13a-2b63-4985-9855-02623bfdf226 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:33.080480 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d7c3e-bc4c-4871-a1a8-9497aee238d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:33.081312 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d7c3e-bc4c-4871-a1a8-9497aee238d8 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.081605 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d7c3e-bc4c-4871-a1a8-9497aee238d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.081887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d7c3e-bc4c-4871-a1a8-9497aee238d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:33.092863 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d7c3e-bc4c-4871-a1a8-9497aee238d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:33.094338 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d7c3e-bc4c-4871-a1a8-9497aee238d8 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:33.111714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-431d7c3e-bc4c-4871-a1a8-9497aee238d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:33.111714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:33.111714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:33.111714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:33.111714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:33.111714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:33.111714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:33.111714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:33.111714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:33.111714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:33.111714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:33.111714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:33.111714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:22:33.111714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:22:33.111714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:22:33.112769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:22:33.112769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:33.112769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:33.112769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:33.112769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:33.112769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:33.112769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:33.112769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:33.112769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:33.112769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:33.112769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:33.112769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:33.112769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:33.112769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:33.112769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:33.112769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:33.112769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:33.113780 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:22:33.113780 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:22:33.113780 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:33.113780 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:33.113780 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 544/1089] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:22:33.118620 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c10e6a-cf01-42ca-8376-fd77f6e94085 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:33.205358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f4d85e-6271-420a-aa3a-3fd5a4738881 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.205647 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f4d85e-6271-420a-aa3a-3fd5a4738881 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.205931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f4d85e-6271-420a-aa3a-3fd5a4738881 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:33.207306 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 545/1090] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:33.212175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741827a-adb5-4bfc-825a-eded0342c3da None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.212487 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741827a-adb5-4bfc-825a-eded0342c3da None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.212751 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741827a-adb5-4bfc-825a-eded0342c3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:33.227058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c10e6a-cf01-42ca-8376-fd77f6e94085 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:33.230757 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-0741827a-adb5-4bfc-825a-eded0342c3da None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:33.231871 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0741827a-adb5-4bfc-825a-eded0342c3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:33.258860 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:33.266951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412334b2-edaf-4721-8a10-4c3e26ec1807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:33.267628 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412334b2-edaf-4721-8a10-4c3e26ec1807 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.267916 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412334b2-edaf-4721-8a10-4c3e26ec1807 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.268198 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412334b2-edaf-4721-8a10-4c3e26ec1807 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:33.268920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412334b2-edaf-4721-8a10-4c3e26ec1807 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:33.269267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412334b2-edaf-4721-8a10-4c3e26ec1807 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:33.270645 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412334b2-edaf-4721-8a10-4c3e26ec1807 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:33.282443 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 546/1091] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:33.288191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412334b2-edaf-4721-8a10-4c3e26ec1807 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:33.299249 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:33.302716 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 546/1092] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:22:33.327454 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741827a-adb5-4bfc-825a-eded0342c3da None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:33.387763 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412334b2-edaf-4721-8a10-4c3e26ec1807 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:33.422711 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475891aa-b16e-4fae-a044-e780010f541c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:33.423503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475891aa-b16e-4fae-a044-e780010f541c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.423628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741827a-adb5-4bfc-825a-eded0342c3da None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:33.423790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475891aa-b16e-4fae-a044-e780010f541c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.424072 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475891aa-b16e-4fae-a044-e780010f541c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:33.431089 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475891aa-b16e-4fae-a044-e780010f541c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:33.432522 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475891aa-b16e-4fae-a044-e780010f541c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:33.454770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-475891aa-b16e-4fae-a044-e780010f541c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:22:33.454770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:33.454770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:33.454770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:33.454770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:33.454770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:33.454770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:33.454770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:33.454770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:33.454770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:33.454770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:33.454770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:33.454770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:22:33.454770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:22:33.454770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:22:33.454770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:22:33.463658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:33.463658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:33.463658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:22:33.463658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:22:33.463658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:22:33.463658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:22:33.463658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:33.463658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:33.463658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:33.463658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:33.463658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:33.463658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:33.463658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:33.463658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:33.463658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:33.463658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:33.463658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:33.463658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:33.465359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:33.465359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:33.465359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:22:33.465359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:22:33.465359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:22:33.465359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:22:33.465359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:22:33.465359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:22:33.465359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:22:33.465359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:33.465359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 547/1093] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jun 16 17:22:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:22:33.466417 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e9728-8d29-45b4-9e8e-3abf5466865b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:33.466417 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e9728-8d29-45b4-9e8e-3abf5466865b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.466678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475891aa-b16e-4fae-a044-e780010f541c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:33.466796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e9728-8d29-45b4-9e8e-3abf5466865b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.466796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e9728-8d29-45b4-9e8e-3abf5466865b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:33.475524 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e9728-8d29-45b4-9e8e-3abf5466865b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:22:33.483772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-193e9728-8d29-45b4-9e8e-3abf5466865b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:22:33.483772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:33.483772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:33.483772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:33.483772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:33.483772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:33.483772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:33.483772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:33.483772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:33.483772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:33.483772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:33.483772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:33.483772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:22:33.483772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:22:33.483772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:22:33.483772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:22:33.484930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:22:33.484930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:22:33.484930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:22:33.484930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:22:33.484930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:33.484930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:33.484930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:33.484930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:33.484930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:33.484930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:33.484930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:33.484930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:33.484930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:33.484930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:33.484930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:33.484930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:33.487053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:33.487053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:33.487053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:33.487053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:33.487053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:22:33.487053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:22:33.487053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:22:33.487053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:22:33.487053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:22:33.487053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:22:33.487053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:22:33.487053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:33.489333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 547/1094] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:22:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:22:33.490802 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e9728-8d29-45b4-9e8e-3abf5466865b None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:33.515862 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:33.532789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:33.568097 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475891aa-b16e-4fae-a044-e780010f541c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:33.593180 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e9728-8d29-45b4-9e8e-3abf5466865b None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:33.609601 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c68828-f4f7-497a-ab67-6f76b727449e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:33.610282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c68828-f4f7-497a-ab67-6f76b727449e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.610567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c68828-f4f7-497a-ab67-6f76b727449e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.610842 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c68828-f4f7-497a-ab67-6f76b727449e None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:33.611680 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c68828-f4f7-497a-ab67-6f76b727449e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:33.611987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c68828-f4f7-497a-ab67-6f76b727449e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:33.613565 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c68828-f4f7-497a-ab67-6f76b727449e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:33.631284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 548/1095] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jun 16 17:22:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:33.631756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771baa16-9451-433a-b2d0-da46f3528eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:33.632453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771baa16-9451-433a-b2d0-da46f3528eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.632748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771baa16-9451-433a-b2d0-da46f3528eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.633032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771baa16-9451-433a-b2d0-da46f3528eb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:33.633830 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771baa16-9451-433a-b2d0-da46f3528eb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:33.634147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771baa16-9451-433a-b2d0-da46f3528eb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:33.635521 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771baa16-9451-433a-b2d0-da46f3528eb3 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:33.637174 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c68828-f4f7-497a-ab67-6f76b727449e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:33.644028 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 548/1096] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:22:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:33.649646 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771baa16-9451-433a-b2d0-da46f3528eb3 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:33.734899 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:33.736344 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c68828-f4f7-497a-ab67-6f76b727449e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:33.751720 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771baa16-9451-433a-b2d0-da46f3528eb3 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:33.762980 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:33.772432 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9567f2e-def7-4395-ab4b-d0231e9f236d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:33.773089 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9567f2e-def7-4395-ab4b-d0231e9f236d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.773410 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9567f2e-def7-4395-ab4b-d0231e9f236d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.773690 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9567f2e-def7-4395-ab4b-d0231e9f236d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:33.785717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ba3ce-0a80-49a0-8df7-81b80725f448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:33.786599 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ba3ce-0a80-49a0-8df7-81b80725f448 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.786886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ba3ce-0a80-49a0-8df7-81b80725f448 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.787166 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ba3ce-0a80-49a0-8df7-81b80725f448 None admin] PATH_INFO: `/v3/users/439ce3ee63a54e8fafcd10d9872821be` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:33.788380 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ba3ce-0a80-49a0-8df7-81b80725f448 None admin] RBAC: Authorizing `identity:get_user(user_id=439ce3ee63a54e8fafcd10d9872821be)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:33.789843 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ba3ce-0a80-49a0-8df7-81b80725f448 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:33.791457 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 549/1097] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:22:33 2022] GET /identity/v3/users/439ce3ee63a54e8fafcd10d9872821be => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:33.798273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ba3ce-0a80-49a0-8df7-81b80725f448 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:33.851998 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9567f2e-def7-4395-ab4b-d0231e9f236d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:33.853390 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9567f2e-def7-4395-ab4b-d0231e9f236d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:33.912405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ba3ce-0a80-49a0-8df7-81b80725f448 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:33.946543 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2816007-5c52-4b76-acea-51c963d804ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:33.947192 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2816007-5c52-4b76-acea-51c963d804ad None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.947491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2816007-5c52-4b76-acea-51c963d804ad None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.947774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2816007-5c52-4b76-acea-51c963d804ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:33.953816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 549/1098] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:22:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 17:22:33.959512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9567f2e-def7-4395-ab4b-d0231e9f236d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:33.971221 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:33.988427 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:34.017886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2816007-5c52-4b76-acea-51c963d804ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:34.019345 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2816007-5c52-4b76-acea-51c963d804ad None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:34.059030 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9567f2e-def7-4395-ab4b-d0231e9f236d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:34.093512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172f76d-7b1b-4243-a269-95365bda2ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:34.094314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172f76d-7b1b-4243-a269-95365bda2ffb None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:34.094598 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172f76d-7b1b-4243-a269-95365bda2ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:34.094876 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172f76d-7b1b-4243-a269-95365bda2ffb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:34.104690 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172f76d-7b1b-4243-a269-95365bda2ffb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:34.106075 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172f76d-7b1b-4243-a269-95365bda2ffb None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:34.119177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 550/1099] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:22:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 17:22:34.124766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-9172f76d-7b1b-4243-a269-95365bda2ffb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:34.124766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:34.124766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:34.124766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:34.124766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:34.124766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:34.124766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:34.124766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:34.124766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:34.124766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:34.124766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:34.124766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:34.124766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:22:34.124766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:22:34.124766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:22:34.125777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:22:34.125777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:34.125777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:34.125777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:34.125777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:34.125777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:34.125777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:34.125777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:34.125777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:34.125777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:34.125777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:34.125777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:34.125777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:34.125777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:34.125777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:34.125777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:34.125777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:34.126709 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:22:34.126709 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:22:34.126709 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:22:34.126709 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:34.126948 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2816007-5c52-4b76-acea-51c963d804ad None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:34.127029 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 550/1100] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:22:34.133741 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172f76d-7b1b-4243-a269-95365bda2ffb None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:34.185208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:34.231383 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:34.237796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2816007-5c52-4b76-acea-51c963d804ad None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:34.255853 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172f76d-7b1b-4243-a269-95365bda2ffb None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:34.272990 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a80cae-7041-44c0-9c70-3fedc245d6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:34.273895 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a80cae-7041-44c0-9c70-3fedc245d6fd None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:34.274187 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a80cae-7041-44c0-9c70-3fedc245d6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:34.274464 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a80cae-7041-44c0-9c70-3fedc245d6fd None admin] PATH_INFO: `/v3/projects/bdd732d58dff46eab9a48d755dbccc92` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:34.275665 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a80cae-7041-44c0-9c70-3fedc245d6fd None admin] RBAC: Authorizing `identity:get_project(project_id=bdd732d58dff46eab9a48d755dbccc92)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:34.277105 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a80cae-7041-44c0-9c70-3fedc245d6fd None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:34.280066 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 551/1101] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:22:34 2022] GET /identity/v3/projects/bdd732d58dff46eab9a48d755dbccc92 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:34.286023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a80cae-7041-44c0-9c70-3fedc245d6fd None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:34.290964 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273dd789-ee2b-4e52-9566-530e43eb3983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:34.291634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dd789-ee2b-4e52-9566-530e43eb3983 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:34.291921 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dd789-ee2b-4e52-9566-530e43eb3983 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:34.292203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dd789-ee2b-4e52-9566-530e43eb3983 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:34.292937 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273dd789-ee2b-4e52-9566-530e43eb3983 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:34.293283 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273dd789-ee2b-4e52-9566-530e43eb3983 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:34.294698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273dd789-ee2b-4e52-9566-530e43eb3983 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:34.305954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 551/1102] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:34.311406 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273dd789-ee2b-4e52-9566-530e43eb3983 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:34.389442 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a80cae-7041-44c0-9c70-3fedc245d6fd None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:34.410245 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:34.414102 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273dd789-ee2b-4e52-9566-530e43eb3983 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:34.425127 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c05604-c384-4411-b178-7e1135ebc33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:34.425828 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c05604-c384-4411-b178-7e1135ebc33b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:34.426109 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c05604-c384-4411-b178-7e1135ebc33b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:34.426396 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c05604-c384-4411-b178-7e1135ebc33b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:34.427247 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c05604-c384-4411-b178-7e1135ebc33b None admin] RBAC: Adding query filter params (role.id=25a16d8ea4084e9eacdebb7a4dcd2337, scope.project.id=bdd732d58dff46eab9a48d755dbccc92, user.id=439ce3ee63a54e8fafcd10d9872821be) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:34.427571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c05604-c384-4411-b178-7e1135ebc33b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:34.429001 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c05604-c384-4411-b178-7e1135ebc33b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:34.435666 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 552/1103] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:22:34 2022] GET /identity/v3/role_assignments?user.id=439ce3ee63a54e8fafcd10d9872821be&scope.project.id=bdd732d58dff46eab9a48d755dbccc92&role.id=25a16d8ea4084e9eacdebb7a4dcd2337 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:22:34.447681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a849939-0c9d-41c6-93c6-d3370623004b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:34.448501 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a849939-0c9d-41c6-93c6-d3370623004b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:34.448786 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a849939-0c9d-41c6-93c6-d3370623004b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:34.449069 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a849939-0c9d-41c6-93c6-d3370623004b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:34.457713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a849939-0c9d-41c6-93c6-d3370623004b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:34.459195 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a849939-0c9d-41c6-93c6-d3370623004b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:34.461322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:34.476028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-1a849939-0c9d-41c6-93c6-d3370623004b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:22:34.476028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:34.476028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:34.476028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:34.476028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:34.476028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:34.476028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:34.476028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:34.476028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:34.476028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:34.476028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:34.476028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:34.476028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:22:34.476028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:22:34.476028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:22:34.477312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:22:34.477312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:34.477312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:34.477312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:34.477312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:34.477312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:34.477312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:34.477312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:34.477312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:34.477312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:34.477312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:34.477312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:34.477312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:34.477312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:34.477312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:34.477312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:34.477312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:34.478272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:22:34.478272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:22:34.478272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:22:34.478272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:22:34.478272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:22:34.478272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:22:34.478272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:22:34.478272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:34.478272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 552/1104] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:22:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:22:34.483377 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c05604-c384-4411-b178-7e1135ebc33b None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:34.595136 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c05604-c384-4411-b178-7e1135ebc33b None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:34.629484 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ebd63-5495-4d4f-861b-8497bb123d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:34.630153 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ebd63-5495-4d4f-861b-8497bb123d4b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:34.630443 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ebd63-5495-4d4f-861b-8497bb123d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:34.630724 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ebd63-5495-4d4f-861b-8497bb123d4b None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:34.631792 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ebd63-5495-4d4f-861b-8497bb123d4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:34.632109 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ebd63-5495-4d4f-861b-8497bb123d4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:34.633577 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ebd63-5495-4d4f-861b-8497bb123d4b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:34.646463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 553/1105] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:22:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:34.652487 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a849939-0c9d-41c6-93c6-d3370623004b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:34.756391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a849939-0c9d-41c6-93c6-d3370623004b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:34.790931 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ff915-fef0-41c3-a18d-3cd443b1de99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:34.791601 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ff915-fef0-41c3-a18d-3cd443b1de99 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:34.791888 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ff915-fef0-41c3-a18d-3cd443b1de99 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:34.792171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ff915-fef0-41c3-a18d-3cd443b1de99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:34.793015 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ff915-fef0-41c3-a18d-3cd443b1de99 None admin] RBAC: Adding query filter params (role.id=3a41c48a11174f87a500cc1d6618ee65, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=689a31e26c6b454a91c3fd79678b4d07) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:34.793371 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ff915-fef0-41c3-a18d-3cd443b1de99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:34.794844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ff915-fef0-41c3-a18d-3cd443b1de99 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:34.800814 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 553/1106] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:22:34 2022] GET /identity/v3/role_assignments?user.id=689a31e26c6b454a91c3fd79678b4d07&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=3a41c48a11174f87a500cc1d6618ee65 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:36.602860 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebe517-ddf2-4145-b7ba-5716bbbac049 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:36.603403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebe517-ddf2-4145-b7ba-5716bbbac049 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:36.603403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebe517-ddf2-4145-b7ba-5716bbbac049 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:36.604846 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 554/1107] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:36.610342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a754f-bfbd-4e48-8e95-7af07b2b4486 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:36.610945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a754f-bfbd-4e48-8e95-7af07b2b4486 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:36.610945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a754f-bfbd-4e48-8e95-7af07b2b4486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:36.627450 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-fe1a754f-bfbd-4e48-8e95-7af07b2b4486 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:36.628633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fe1a754f-bfbd-4e48-8e95-7af07b2b4486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:36.684486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:36.694816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 554/1108] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:36.720534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebe517-ddf2-4145-b7ba-5716bbbac049 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:36.767102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:36.821831 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebe517-ddf2-4145-b7ba-5716bbbac049 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:36.856147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98317b7-6307-4438-9c7e-95adb108c9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:36.857082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98317b7-6307-4438-9c7e-95adb108c9af None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:36.857235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98317b7-6307-4438-9c7e-95adb108c9af None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:36.857502 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98317b7-6307-4438-9c7e-95adb108c9af None admin] PATH_INFO: `/v3/domains/default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:36.858685 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98317b7-6307-4438-9c7e-95adb108c9af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:36.860107 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98317b7-6307-4438-9c7e-95adb108c9af None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:36.861743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 555/1109] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:36.867660 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a754f-bfbd-4e48-8e95-7af07b2b4486 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:36.990201 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a754f-bfbd-4e48-8e95-7af07b2b4486 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:37.035385 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e1c65-f9a5-4423-adf4-8ff68483851a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:37.036062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e1c65-f9a5-4423-adf4-8ff68483851a None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:37.036352 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e1c65-f9a5-4423-adf4-8ff68483851a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:37.036633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e1c65-f9a5-4423-adf4-8ff68483851a None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:37.037373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e1c65-f9a5-4423-adf4-8ff68483851a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:37.038811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e1c65-f9a5-4423-adf4-8ff68483851a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:37.057900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 555/1110] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 17:22:36 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:39.218942 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9895d7-1f44-4687-8e6a-f3e2f34b51db None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:39.219509 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9895d7-1f44-4687-8e6a-f3e2f34b51db None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:39.219509 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9895d7-1f44-4687-8e6a-f3e2f34b51db None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:39.220895 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 556/1111] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:39.226045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301f28c-457a-44bb-ad2e-ea9c01259bb4 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:39.226454 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301f28c-457a-44bb-ad2e-ea9c01259bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:39.226615 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301f28c-457a-44bb-ad2e-ea9c01259bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:39.243584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-c301f28c-457a-44bb-ad2e-ea9c01259bb4 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:39.244776 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c301f28c-457a-44bb-ad2e-ea9c01259bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:39.288939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:39.312848 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 556/1112] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:39.336648 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9895d7-1f44-4687-8e6a-f3e2f34b51db None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:39.439005 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9895d7-1f44-4687-8e6a-f3e2f34b51db None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:39.451895 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:39.472494 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7ece3-a3c5-4f37-8224-46a1dd559ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:39.473319 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7ece3-a3c5-4f37-8224-46a1dd559ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:39.473613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7ece3-a3c5-4f37-8224-46a1dd559ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:39.473899 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7ece3-a3c5-4f37-8224-46a1dd559ed6 None admin] PATH_INFO: `/v3/roles/1423a24997e840608b6c522958481fde` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:39.475366 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7ece3-a3c5-4f37-8224-46a1dd559ed6 None admin] RBAC: Authorizing `identity:get_role(role_id=1423a24997e840608b6c522958481fde)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:39.476739 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7ece3-a3c5-4f37-8224-46a1dd559ed6 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:39.478211 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 557/1113] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:22:39 2022] GET /identity/v3/roles/1423a24997e840608b6c522958481fde => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:39.484533 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301f28c-457a-44bb-ad2e-ea9c01259bb4 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:39.581353 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301f28c-457a-44bb-ad2e-ea9c01259bb4 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:39.614310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03069900-c344-402e-8354-49f5efad89dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:39.615185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03069900-c344-402e-8354-49f5efad89dd None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:39.615469 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03069900-c344-402e-8354-49f5efad89dd None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:39.615749 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03069900-c344-402e-8354-49f5efad89dd None admin] PATH_INFO: `/v3/users/50adbdcfa8ae4f91914292db3b2b05d2` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:39.629422 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03069900-c344-402e-8354-49f5efad89dd None admin] RBAC: Authorizing `identity:get_user(user_id=50adbdcfa8ae4f91914292db3b2b05d2)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:39.630865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03069900-c344-402e-8354-49f5efad89dd None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:39.632780 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 557/1114] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:22:39 2022] GET /identity/v3/users/50adbdcfa8ae4f91914292db3b2b05d2 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:39.638491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7ece3-a3c5-4f37-8224-46a1dd559ed6 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:39.736613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7ece3-a3c5-4f37-8224-46a1dd559ed6 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:39.769988 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a2ab3-75ab-422e-acb9-ae73d7642e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:39.770628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a2ab3-75ab-422e-acb9-ae73d7642e5f None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:39.770912 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a2ab3-75ab-422e-acb9-ae73d7642e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:39.771191 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a2ab3-75ab-422e-acb9-ae73d7642e5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:39.817629 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:39.839396 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a2ab3-75ab-422e-acb9-ae73d7642e5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:39.841563 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a2ab3-75ab-422e-acb9-ae73d7642e5f None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:39.936606 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 558/1115] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:22:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:22:39.942526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03069900-c344-402e-8354-49f5efad89dd None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:39.975739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:40.037199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03069900-c344-402e-8354-49f5efad89dd None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:40.069681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620adcc1-3a8a-49f5-97d0-753d609be4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:40.070535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620adcc1-3a8a-49f5-97d0-753d609be4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:40.070818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620adcc1-3a8a-49f5-97d0-753d609be4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:40.071096 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620adcc1-3a8a-49f5-97d0-753d609be4f8 None admin] PATH_INFO: `/v3/projects/bdd732d58dff46eab9a48d755dbccc92` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:40.072264 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620adcc1-3a8a-49f5-97d0-753d609be4f8 None admin] RBAC: Authorizing `identity:get_project(project_id=bdd732d58dff46eab9a48d755dbccc92)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:40.073722 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620adcc1-3a8a-49f5-97d0-753d609be4f8 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:40.075456 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 558/1116] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:22:39 2022] GET /identity/v3/projects/bdd732d58dff46eab9a48d755dbccc92 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:40.081528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a2ab3-75ab-422e-acb9-ae73d7642e5f None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:40.160462 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:40.180310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a2ab3-75ab-422e-acb9-ae73d7642e5f None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:40.212911 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d58e8-7331-4bcc-b238-c1b7a560f52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:40.213586 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d58e8-7331-4bcc-b238-c1b7a560f52f None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:40.213875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d58e8-7331-4bcc-b238-c1b7a560f52f None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:40.214152 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d58e8-7331-4bcc-b238-c1b7a560f52f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:40.214977 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d58e8-7331-4bcc-b238-c1b7a560f52f None admin] RBAC: Adding query filter params (role.id=1423a24997e840608b6c522958481fde, scope.project.id=bdd732d58dff46eab9a48d755dbccc92, user.id=50adbdcfa8ae4f91914292db3b2b05d2) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:40.215293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d58e8-7331-4bcc-b238-c1b7a560f52f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:40.217454 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d58e8-7331-4bcc-b238-c1b7a560f52f None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:40.222767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 559/1117] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:22:40 2022] GET /identity/v3/role_assignments?user.id=50adbdcfa8ae4f91914292db3b2b05d2&scope.project.id=bdd732d58dff46eab9a48d755dbccc92&role.id=1423a24997e840608b6c522958481fde => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:42.333203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06c312-acdc-42a9-9e4d-a99a8dd672bc None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:42.333749 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06c312-acdc-42a9-9e4d-a99a8dd672bc None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:42.333749 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06c312-acdc-42a9-9e4d-a99a8dd672bc None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:42.335225 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 559/1118] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:42.341281 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19336c61-1dcd-48ee-934c-a35847cf1a97 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:42.341787 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19336c61-1dcd-48ee-934c-a35847cf1a97 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:42.341933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19336c61-1dcd-48ee-934c-a35847cf1a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:42.360338 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-19336c61-1dcd-48ee-934c-a35847cf1a97 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:42.361569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-19336c61-1dcd-48ee-934c-a35847cf1a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:42.434176 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 560/1119] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:42.456900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06c312-acdc-42a9-9e4d-a99a8dd672bc None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:42.557621 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06c312-acdc-42a9-9e4d-a99a8dd672bc None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:42.560128 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:42.596128 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3c6e1-82fb-477b-aa0b-e83d52b86aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:42.596992 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3c6e1-82fb-477b-aa0b-e83d52b86aaf None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:42.597321 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3c6e1-82fb-477b-aa0b-e83d52b86aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:42.597606 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3c6e1-82fb-477b-aa0b-e83d52b86aaf None admin] PATH_INFO: `/v3/roles/1423a24997e840608b6c522958481fde` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:42.599105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3c6e1-82fb-477b-aa0b-e83d52b86aaf None admin] RBAC: Authorizing `identity:get_role(role_id=1423a24997e840608b6c522958481fde)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:42.600477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3c6e1-82fb-477b-aa0b-e83d52b86aaf None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:42.601897 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 560/1120] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:22:42 2022] GET /identity/v3/roles/1423a24997e840608b6c522958481fde => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:42.608205 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19336c61-1dcd-48ee-934c-a35847cf1a97 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:42.682952 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:42.708494 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19336c61-1dcd-48ee-934c-a35847cf1a97 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:42.742406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc68059-b4ad-4c17-90ee-052e2d43cda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:42.743278 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc68059-b4ad-4c17-90ee-052e2d43cda0 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:42.743569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc68059-b4ad-4c17-90ee-052e2d43cda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:42.743947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc68059-b4ad-4c17-90ee-052e2d43cda0 None admin] PATH_INFO: `/v3/users/50adbdcfa8ae4f91914292db3b2b05d2` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:42.745138 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc68059-b4ad-4c17-90ee-052e2d43cda0 None admin] RBAC: Authorizing `identity:get_user(user_id=50adbdcfa8ae4f91914292db3b2b05d2)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:42.746607 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc68059-b4ad-4c17-90ee-052e2d43cda0 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:42.748351 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 561/1121] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:22:42 2022] GET /identity/v3/users/50adbdcfa8ae4f91914292db3b2b05d2 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:42.754114 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3c6e1-82fb-477b-aa0b-e83d52b86aaf None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:42.851826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3c6e1-82fb-477b-aa0b-e83d52b86aaf None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:42.885027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d856b50-013f-4659-8e0c-21bcf96a8bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:42.885913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d856b50-013f-4659-8e0c-21bcf96a8bcd None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:42.886203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d856b50-013f-4659-8e0c-21bcf96a8bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:42.886485 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d856b50-013f-4659-8e0c-21bcf96a8bcd None admin] PATH_INFO: `/v3/projects/bdd732d58dff46eab9a48d755dbccc92` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:42.888454 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d856b50-013f-4659-8e0c-21bcf96a8bcd None admin] RBAC: Authorizing `identity:get_project(project_id=bdd732d58dff46eab9a48d755dbccc92)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:42.890003 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d856b50-013f-4659-8e0c-21bcf96a8bcd None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:42.891707 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 561/1122] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:22:42 2022] GET /identity/v3/projects/bdd732d58dff46eab9a48d755dbccc92 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:42.897662 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc68059-b4ad-4c17-90ee-052e2d43cda0 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:42.994422 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc68059-b4ad-4c17-90ee-052e2d43cda0 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:43.028739 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c372a-4ba9-4ede-bac0-e7ad47e97c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:43.029718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c372a-4ba9-4ede-bac0-e7ad47e97c2f None admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:43.030000 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c372a-4ba9-4ede-bac0-e7ad47e97c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:43.030287 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c372a-4ba9-4ede-bac0-e7ad47e97c2f None admin] PATH_INFO: `/v3/projects/bdd732d58dff46eab9a48d755dbccc92/users/50adbdcfa8ae4f91914292db3b2b05d2/roles/1423a24997e840608b6c522958481fde` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:43.032450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c372a-4ba9-4ede-bac0-e7ad47e97c2f None admin] RBAC: Authorizing `identity:create_grant(project_id=bdd732d58dff46eab9a48d755dbccc92, user_id=50adbdcfa8ae4f91914292db3b2b05d2, role_id=1423a24997e840608b6c522958481fde)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:43.033894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c372a-4ba9-4ede-bac0-e7ad47e97c2f None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:43.046371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 562/1123] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:22:42 2022] PUT /identity/v3/projects/bdd732d58dff46eab9a48d755dbccc92/users/50adbdcfa8ae4f91914292db3b2b05d2/roles/1423a24997e840608b6c522958481fde => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:45.129382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ed11c-5a00-4540-9470-a1cf3fc028a0 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:45.129966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ed11c-5a00-4540-9470-a1cf3fc028a0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:45.129966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ed11c-5a00-4540-9470-a1cf3fc028a0 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:45.131586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 562/1124] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:45.136638 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ac4ef-8dbb-410b-b289-d9c247246de7 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:45.137042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ac4ef-8dbb-410b-b289-d9c247246de7 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:45.137262 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ac4ef-8dbb-410b-b289-d9c247246de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:45.142385 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:45.155930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-1c2ac4ef-8dbb-410b-b289-d9c247246de7 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:45.157115 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-1c2ac4ef-8dbb-410b-b289-d9c247246de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:45.221660 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 563/1125] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:45.245816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ed11c-5a00-4540-9470-a1cf3fc028a0 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:45.289949 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:45.349731 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ed11c-5a00-4540-9470-a1cf3fc028a0 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:45.387646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2da466-b0da-4268-9c60-a280aaf2054e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:45.389300 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2da466-b0da-4268-9c60-a280aaf2054e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:45.389594 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2da466-b0da-4268-9c60-a280aaf2054e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:45.389877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2da466-b0da-4268-9c60-a280aaf2054e None admin] PATH_INFO: `/v3/roles/1423a24997e840608b6c522958481fde` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:45.391386 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2da466-b0da-4268-9c60-a280aaf2054e None admin] RBAC: Authorizing `identity:get_role(role_id=1423a24997e840608b6c522958481fde)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:45.392757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2da466-b0da-4268-9c60-a280aaf2054e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:45.394194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 563/1126] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:22:45 2022] GET /identity/v3/roles/1423a24997e840608b6c522958481fde => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:45.399952 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ac4ef-8dbb-410b-b289-d9c247246de7 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:45.500495 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ac4ef-8dbb-410b-b289-d9c247246de7 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:45.530144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:45.535642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d69888-aab1-4bb3-8a7b-16e25f293033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:45.536520 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d69888-aab1-4bb3-8a7b-16e25f293033 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:45.536811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d69888-aab1-4bb3-8a7b-16e25f293033 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:45.537089 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d69888-aab1-4bb3-8a7b-16e25f293033 None admin] PATH_INFO: `/v3/users/50adbdcfa8ae4f91914292db3b2b05d2` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:45.538307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d69888-aab1-4bb3-8a7b-16e25f293033 None admin] RBAC: Authorizing `identity:get_user(user_id=50adbdcfa8ae4f91914292db3b2b05d2)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:45.539733 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d69888-aab1-4bb3-8a7b-16e25f293033 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:45.541396 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 564/1127] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:22:45 2022] GET /identity/v3/users/50adbdcfa8ae4f91914292db3b2b05d2 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:45.547221 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2da466-b0da-4268-9c60-a280aaf2054e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:45.645989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2da466-b0da-4268-9c60-a280aaf2054e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:45.655516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:45.679867 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3515422-a7a8-4c3f-ad75-d931fc7593de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:45.680515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3515422-a7a8-4c3f-ad75-d931fc7593de None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:45.680804 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3515422-a7a8-4c3f-ad75-d931fc7593de None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:45.681084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3515422-a7a8-4c3f-ad75-d931fc7593de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:45.749961 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3515422-a7a8-4c3f-ad75-d931fc7593de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:45.751420 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3515422-a7a8-4c3f-ad75-d931fc7593de None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:45.847153 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 564/1128] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:22:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:22:45.853469 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d69888-aab1-4bb3-8a7b-16e25f293033 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:45.951090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d69888-aab1-4bb3-8a7b-16e25f293033 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:45.986130 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37491e0-884b-4fc2-a168-b8fb155fc4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:45.986988 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37491e0-884b-4fc2-a168-b8fb155fc4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:45.987278 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37491e0-884b-4fc2-a168-b8fb155fc4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:45.987563 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37491e0-884b-4fc2-a168-b8fb155fc4d0 None admin] PATH_INFO: `/v3/projects/bdd732d58dff46eab9a48d755dbccc92` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:45.988741 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37491e0-884b-4fc2-a168-b8fb155fc4d0 None admin] RBAC: Authorizing `identity:get_project(project_id=bdd732d58dff46eab9a48d755dbccc92)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:45.990216 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37491e0-884b-4fc2-a168-b8fb155fc4d0 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:45.991929 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 565/1129] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:22:45 2022] GET /identity/v3/projects/bdd732d58dff46eab9a48d755dbccc92 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:45.999159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3515422-a7a8-4c3f-ad75-d931fc7593de None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:46.009375 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:46.096568 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3515422-a7a8-4c3f-ad75-d931fc7593de None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:46.128854 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa633fd-ad51-41f4-a9b8-0b53ebb2e424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:46.129520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa633fd-ad51-41f4-a9b8-0b53ebb2e424 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:46.130079 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa633fd-ad51-41f4-a9b8-0b53ebb2e424 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:46.130079 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa633fd-ad51-41f4-a9b8-0b53ebb2e424 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:46.130878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa633fd-ad51-41f4-a9b8-0b53ebb2e424 None admin] RBAC: Adding query filter params (role.id=1423a24997e840608b6c522958481fde, scope.project.id=bdd732d58dff46eab9a48d755dbccc92, user.id=50adbdcfa8ae4f91914292db3b2b05d2) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:46.131447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa633fd-ad51-41f4-a9b8-0b53ebb2e424 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:46.132865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa633fd-ad51-41f4-a9b8-0b53ebb2e424 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:46.139211 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 565/1130] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:22:45 2022] GET /identity/v3/role_assignments?user.id=50adbdcfa8ae4f91914292db3b2b05d2&scope.project.id=bdd732d58dff46eab9a48d755dbccc92&role.id=1423a24997e840608b6c522958481fde => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:48.279564 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2f831-134a-4c3e-b0b5-03a7a1fcc76d None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:48.280118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2f831-134a-4c3e-b0b5-03a7a1fcc76d None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:48.280118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2f831-134a-4c3e-b0b5-03a7a1fcc76d None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:48.281544 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 566/1131] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:48.286598 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447eae6-cde2-4532-abe2-a76294fbfbc2 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:48.287014 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447eae6-cde2-4532-abe2-a76294fbfbc2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:48.287154 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447eae6-cde2-4532-abe2-a76294fbfbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:48.305342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-7447eae6-cde2-4532-abe2-a76294fbfbc2 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:48.306530 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7447eae6-cde2-4532-abe2-a76294fbfbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:48.372435 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 566/1132] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:48.395817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2f831-134a-4c3e-b0b5-03a7a1fcc76d None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:48.463033 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:48.498240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2f831-134a-4c3e-b0b5-03a7a1fcc76d None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:48.532819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c31ab7-b1a1-4ea2-b5eb-b09d1d7a2b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:48.533642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c31ab7-b1a1-4ea2-b5eb-b09d1d7a2b93 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:48.533936 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c31ab7-b1a1-4ea2-b5eb-b09d1d7a2b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:48.534223 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c31ab7-b1a1-4ea2-b5eb-b09d1d7a2b93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:48.535401 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c31ab7-b1a1-4ea2-b5eb-b09d1d7a2b93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:48.536832 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c31ab7-b1a1-4ea2-b5eb-b09d1d7a2b93 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:48.538488 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 567/1133] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:48.544449 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447eae6-cde2-4532-abe2-a76294fbfbc2 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:48.546644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:48.645223 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447eae6-cde2-4532-abe2-a76294fbfbc2 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:48.678320 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce401e87-2fab-4d36-8e58-cfaa4ebab49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:48.678983 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce401e87-2fab-4d36-8e58-cfaa4ebab49d None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:48.679272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce401e87-2fab-4d36-8e58-cfaa4ebab49d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:48.679553 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce401e87-2fab-4d36-8e58-cfaa4ebab49d None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:48.680292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce401e87-2fab-4d36-8e58-cfaa4ebab49d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:48.681747 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce401e87-2fab-4d36-8e58-cfaa4ebab49d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:48.707335 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 567/1134] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 17:22:48 2022] POST /identity/v3/projects => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:50.885102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe913289-c5e8-4bad-ab72-bb9239cbec48 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:50.885672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe913289-c5e8-4bad-ab72-bb9239cbec48 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:50.885791 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe913289-c5e8-4bad-ab72-bb9239cbec48 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:50.887210 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 568/1135] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:50.892612 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16de34-da90-4448-aca3-e6b7e4ad6a3d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:50.893052 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16de34-da90-4448-aca3-e6b7e4ad6a3d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:50.893206 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16de34-da90-4448-aca3-e6b7e4ad6a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:50.909801 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1f16de34-da90-4448-aca3-e6b7e4ad6a3d None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:50.911012 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1f16de34-da90-4448-aca3-e6b7e4ad6a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:50.961293 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:50.979167 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 568/1136] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:51.004906 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe913289-c5e8-4bad-ab72-bb9239cbec48 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:51.110866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe913289-c5e8-4bad-ab72-bb9239cbec48 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:51.144889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08215cf5-45da-41da-b24e-7143e5428000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:51.145635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08215cf5-45da-41da-b24e-7143e5428000 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:51.145923 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08215cf5-45da-41da-b24e-7143e5428000 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:51.146200 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08215cf5-45da-41da-b24e-7143e5428000 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:51.147368 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08215cf5-45da-41da-b24e-7143e5428000 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:51.148799 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08215cf5-45da-41da-b24e-7143e5428000 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:51.150436 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 569/1137] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:22:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:51.156784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16de34-da90-4448-aca3-e6b7e4ad6a3d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:51.253492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16de34-da90-4448-aca3-e6b7e4ad6a3d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:51.286823 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41803d2-c082-46c5-a3c4-fb3e9c538f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:51.287497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41803d2-c082-46c5-a3c4-fb3e9c538f97 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:51.287784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41803d2-c082-46c5-a3c4-fb3e9c538f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:51.288062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41803d2-c082-46c5-a3c4-fb3e9c538f97 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:51.288774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41803d2-c082-46c5-a3c4-fb3e9c538f97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:51.290240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41803d2-c082-46c5-a3c4-fb3e9c538f97 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:51.311870 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 569/1138] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 17:22:51 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:53.476677 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc83473-f339-41a6-ad07-6183903ad482 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:53.477302 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc83473-f339-41a6-ad07-6183903ad482 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:53.477302 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc83473-f339-41a6-ad07-6183903ad482 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:53.478637 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 570/1139] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:53.483871 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969245b7-5be0-4e7f-a588-80fead892ae2 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:53.484305 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969245b7-5be0-4e7f-a588-80fead892ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:53.484438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969245b7-5be0-4e7f-a588-80fead892ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:53.501482 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-969245b7-5be0-4e7f-a588-80fead892ae2 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:53.502600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-969245b7-5be0-4e7f-a588-80fead892ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:53.536294 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:53.569694 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 570/1140] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:53.593411 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc83473-f339-41a6-ad07-6183903ad482 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:53.597217 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:53.693225 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc83473-f339-41a6-ad07-6183903ad482 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:53.725613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1597a1-8fe8-4347-ba38-5d08429ca6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:53.726399 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1597a1-8fe8-4347-ba38-5d08429ca6af None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:53.726689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1597a1-8fe8-4347-ba38-5d08429ca6af None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:53.726972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1597a1-8fe8-4347-ba38-5d08429ca6af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:53.737510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1597a1-8fe8-4347-ba38-5d08429ca6af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:22:53.745110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-ad1597a1-8fe8-4347-ba38-5d08429ca6af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:22:53.745110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:53.745110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:53.745110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:53.745110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:53.745110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:53.745110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:53.745110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:53.745110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:53.745110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:53.745110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:53.745110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:53.745110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:22:53.745110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:22:53.745110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:22:53.745110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:22:53.746910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:22:53.746910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:22:53.746910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:22:53.746910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:22:53.746910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:53.746910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:53.746910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:53.746910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:53.746910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:53.746910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:53.746910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:53.746910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:53.746910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:53.746910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:53.746910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:53.746910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:53.748346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:53.748346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:53.748346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:53.748346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:53.748346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:22:53.748346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:22:53.748346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:22:53.748346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:22:53.748346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:22:53.748346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:22:53.748346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:22:53.748346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:53.748346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 571/1141] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:22:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:53.752997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969245b7-5be0-4e7f-a588-80fead892ae2 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:53.854563 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969245b7-5be0-4e7f-a588-80fead892ae2 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:53.888194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c56de6-4b0f-491d-9010-ca065e597c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:53.888854 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c56de6-4b0f-491d-9010-ca065e597c80 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:53.889136 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c56de6-4b0f-491d-9010-ca065e597c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:53.889451 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c56de6-4b0f-491d-9010-ca065e597c80 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:53.890194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c56de6-4b0f-491d-9010-ca065e597c80 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:53.890502 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c56de6-4b0f-491d-9010-ca065e597c80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:53.891855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c56de6-4b0f-491d-9010-ca065e597c80 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:53.899624 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 571/1142] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:22:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:53.905952 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1597a1-8fe8-4347-ba38-5d08429ca6af None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:53.966914 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:54.005095 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1597a1-8fe8-4347-ba38-5d08429ca6af None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:54.037912 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e96d17-9757-4e7d-814c-9b2e9b82d853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:54.038780 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e96d17-9757-4e7d-814c-9b2e9b82d853 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:54.039070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e96d17-9757-4e7d-814c-9b2e9b82d853 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:54.039351 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e96d17-9757-4e7d-814c-9b2e9b82d853 None admin] PATH_INFO: `/v3/users/e9d194a6cf404ad8b31e3768f2961f64` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:54.052714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e96d17-9757-4e7d-814c-9b2e9b82d853 None admin] RBAC: Authorizing `identity:get_user(user_id=e9d194a6cf404ad8b31e3768f2961f64)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:54.054181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e96d17-9757-4e7d-814c-9b2e9b82d853 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:54.055822 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 572/1143] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:22:53 2022] GET /identity/v3/users/e9d194a6cf404ad8b31e3768f2961f64 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:54.061600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c56de6-4b0f-491d-9010-ca065e597c80 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:54.093435 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:54.159276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c56de6-4b0f-491d-9010-ca065e597c80 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:54.191791 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28d91e-a90c-40e1-a9aa-b6672ef5fe93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:54.192446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28d91e-a90c-40e1-a9aa-b6672ef5fe93 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:54.192740 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28d91e-a90c-40e1-a9aa-b6672ef5fe93 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:54.193020 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28d91e-a90c-40e1-a9aa-b6672ef5fe93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:54.259785 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28d91e-a90c-40e1-a9aa-b6672ef5fe93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:54.261200 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28d91e-a90c-40e1-a9aa-b6672ef5fe93 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:54.290434 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:54.358596 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 572/1144] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:22:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:22:54.364465 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e96d17-9757-4e7d-814c-9b2e9b82d853 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:54.463092 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e96d17-9757-4e7d-814c-9b2e9b82d853 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:54.485892 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:54.498160 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54b11a-1348-446a-9659-b1ca183b7354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:54.499017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54b11a-1348-446a-9659-b1ca183b7354 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:54.499305 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54b11a-1348-446a-9659-b1ca183b7354 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:54.499586 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54b11a-1348-446a-9659-b1ca183b7354 None admin] PATH_INFO: `/v3/projects/28304d8551614c85a8e45979874cbfcc` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:54.500759 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54b11a-1348-446a-9659-b1ca183b7354 None admin] RBAC: Authorizing `identity:get_project(project_id=28304d8551614c85a8e45979874cbfcc)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:54.502226 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54b11a-1348-446a-9659-b1ca183b7354 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:54.503925 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 573/1145] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:22:54 2022] GET /identity/v3/projects/28304d8551614c85a8e45979874cbfcc => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:54.510298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28d91e-a90c-40e1-a9aa-b6672ef5fe93 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:54.608347 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28d91e-a90c-40e1-a9aa-b6672ef5fe93 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:54.636402 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:54.642351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4062a6-d09f-44ce-8be8-897b7a196730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:54.643007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4062a6-d09f-44ce-8be8-897b7a196730 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:54.643289 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4062a6-d09f-44ce-8be8-897b7a196730 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:54.643564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4062a6-d09f-44ce-8be8-897b7a196730 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:54.644393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4062a6-d09f-44ce-8be8-897b7a196730 None admin] RBAC: Adding query filter params (role.id=25a16d8ea4084e9eacdebb7a4dcd2337, scope.project.id=28304d8551614c85a8e45979874cbfcc, user.id=e9d194a6cf404ad8b31e3768f2961f64) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:54.644706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4062a6-d09f-44ce-8be8-897b7a196730 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:54.646159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4062a6-d09f-44ce-8be8-897b7a196730 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:54.653268 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 573/1146] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:22:54 2022] GET /identity/v3/role_assignments?user.id=e9d194a6cf404ad8b31e3768f2961f64&scope.project.id=28304d8551614c85a8e45979874cbfcc&role.id=25a16d8ea4084e9eacdebb7a4dcd2337 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:56.728083 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa63383-2a09-4ca8-be8d-fe226e365eca None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:56.728619 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa63383-2a09-4ca8-be8d-fe226e365eca None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:56.728714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa63383-2a09-4ca8-be8d-fe226e365eca None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:56.730082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 574/1147] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:56.735877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4cacc-e6f6-43ce-92f1-2d7ebc0573da None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:56.736279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4cacc-e6f6-43ce-92f1-2d7ebc0573da None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:56.736439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4cacc-e6f6-43ce-92f1-2d7ebc0573da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:56.753433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-b0d4cacc-e6f6-43ce-92f1-2d7ebc0573da None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:56.754631 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b0d4cacc-e6f6-43ce-92f1-2d7ebc0573da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:56.820101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 574/1148] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:56.842054 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa63383-2a09-4ca8-be8d-fe226e365eca None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:56.942109 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa63383-2a09-4ca8-be8d-fe226e365eca None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:56.974362 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d5f62-6723-4677-bc3f-1c266d2b73c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:56.975146 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d5f62-6723-4677-bc3f-1c266d2b73c8 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:56.975432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d5f62-6723-4677-bc3f-1c266d2b73c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:56.975706 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d5f62-6723-4677-bc3f-1c266d2b73c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:56.986561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d5f62-6723-4677-bc3f-1c266d2b73c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:22:56.994074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-474d5f62-6723-4677-bc3f-1c266d2b73c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:22:56.994074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:56.994074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:56.994074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:56.994074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:56.994074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:56.994074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:56.994074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:56.994074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:56.994074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:56.994074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:56.994074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:56.994074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:22:56.994074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:22:56.994074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:22:56.994074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:22:56.995694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:22:56.995694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:22:56.995694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:22:56.995694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:22:56.995694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:56.995694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:56.995694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:56.995694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:56.995694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:56.995694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:56.995694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:56.995694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:56.995694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:56.995694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:56.995694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:56.995694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:56.998077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:56.998077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:56.998077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:56.998077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:56.998077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:22:56.998077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:22:56.998077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:22:56.998077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:22:56.998077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:22:56.998077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:22:56.998077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:22:56.998077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:22:56.998077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 575/1149] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:22:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:57.001952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4cacc-e6f6-43ce-92f1-2d7ebc0573da None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:57.099572 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4cacc-e6f6-43ce-92f1-2d7ebc0573da None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:57.125056 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:57.136392 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad49419-959c-4d71-8d16-dea93a1bae56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:57.136392 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad49419-959c-4d71-8d16-dea93a1bae56 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:57.136392 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad49419-959c-4d71-8d16-dea93a1bae56 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:57.136392 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad49419-959c-4d71-8d16-dea93a1bae56 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:57.137017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad49419-959c-4d71-8d16-dea93a1bae56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:57.137017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad49419-959c-4d71-8d16-dea93a1bae56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:57.138153 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad49419-959c-4d71-8d16-dea93a1bae56 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:57.151244 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 575/1150] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:22:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:57.157878 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d5f62-6723-4677-bc3f-1c266d2b73c8 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:57.208885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:57.260843 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d5f62-6723-4677-bc3f-1c266d2b73c8 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:57.294395 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c59c5f-89e5-422f-b358-aefc1edfe12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:57.295265 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c59c5f-89e5-422f-b358-aefc1edfe12d None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:57.295562 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c59c5f-89e5-422f-b358-aefc1edfe12d None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:57.295848 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c59c5f-89e5-422f-b358-aefc1edfe12d None admin] PATH_INFO: `/v3/users/e9d194a6cf404ad8b31e3768f2961f64` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:57.297029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c59c5f-89e5-422f-b358-aefc1edfe12d None admin] RBAC: Authorizing `identity:get_user(user_id=e9d194a6cf404ad8b31e3768f2961f64)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:57.298497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c59c5f-89e5-422f-b358-aefc1edfe12d None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:57.300132 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 576/1151] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:22:57 2022] GET /identity/v3/users/e9d194a6cf404ad8b31e3768f2961f64 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:57.305931 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad49419-959c-4d71-8d16-dea93a1bae56 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:57.402926 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad49419-959c-4d71-8d16-dea93a1bae56 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:57.435397 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee06a38-8562-4eaa-aed5-0a1c7d010e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:57.436247 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee06a38-8562-4eaa-aed5-0a1c7d010e0d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:57.436532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee06a38-8562-4eaa-aed5-0a1c7d010e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:57.436808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee06a38-8562-4eaa-aed5-0a1c7d010e0d None admin] PATH_INFO: `/v3/projects/28304d8551614c85a8e45979874cbfcc` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:57.438001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee06a38-8562-4eaa-aed5-0a1c7d010e0d None admin] RBAC: Authorizing `identity:get_project(project_id=28304d8551614c85a8e45979874cbfcc)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:57.439416 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee06a38-8562-4eaa-aed5-0a1c7d010e0d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:57.441098 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 576/1152] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:22:57 2022] GET /identity/v3/projects/28304d8551614c85a8e45979874cbfcc => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:57.447015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c59c5f-89e5-422f-b358-aefc1edfe12d None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:57.560578 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c59c5f-89e5-422f-b358-aefc1edfe12d None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:57.578628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:57.595411 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf5cfb-aea2-43cb-87f9-82548735da54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:57.596300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf5cfb-aea2-43cb-87f9-82548735da54 None admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:57.596587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf5cfb-aea2-43cb-87f9-82548735da54 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:57.596869 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf5cfb-aea2-43cb-87f9-82548735da54 None admin] PATH_INFO: `/v3/projects/28304d8551614c85a8e45979874cbfcc/users/e9d194a6cf404ad8b31e3768f2961f64/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:57.598833 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf5cfb-aea2-43cb-87f9-82548735da54 None admin] RBAC: Authorizing `identity:create_grant(project_id=28304d8551614c85a8e45979874cbfcc, user_id=e9d194a6cf404ad8b31e3768f2961f64, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:57.600262 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf5cfb-aea2-43cb-87f9-82548735da54 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:57.613019 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 577/1153] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:22:57 2022] PUT /identity/v3/projects/28304d8551614c85a8e45979874cbfcc/users/e9d194a6cf404ad8b31e3768f2961f64/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:59.692741 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a85c22-82df-4cd7-ad21-7885551d8694 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:59.693330 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a85c22-82df-4cd7-ad21-7885551d8694 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:59.693330 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a85c22-82df-4cd7-ad21-7885551d8694 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:59.694731 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 577/1154] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:59.700263 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92ba73-f087-48ee-a69f-d8be8a49306a None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:59.700686 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92ba73-f087-48ee-a69f-d8be8a49306a None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:59.700824 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92ba73-f087-48ee-a69f-d8be8a49306a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:59.717948 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-6c92ba73-f087-48ee-a69f-d8be8a49306a None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:59.719117 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-6c92ba73-f087-48ee-a69f-d8be8a49306a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:59.784582 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 578/1155] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:22:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:22:59.807517 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a85c22-82df-4cd7-ad21-7885551d8694 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:59.865352 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:59.909665 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a85c22-82df-4cd7-ad21-7885551d8694 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:59.941864 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aadf1a9-ad77-402d-b78d-dddc605a4705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:59.942652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadf1a9-ad77-402d-b78d-dddc605a4705 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:59.942988 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadf1a9-ad77-402d-b78d-dddc605a4705 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:59.943223 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadf1a9-ad77-402d-b78d-dddc605a4705 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:59.953619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aadf1a9-ad77-402d-b78d-dddc605a4705 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:22:59.961248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-6aadf1a9-ad77-402d-b78d-dddc605a4705 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:22:59.961248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:22:59.961248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:22:59.961248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:22:59.961248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:22:59.961248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:22:59.961248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:22:59.961248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:22:59.961248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:22:59.961248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:22:59.961248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:22:59.961248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:22:59.961248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:22:59.961248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:22:59.961248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:22:59.961248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:22:59.962373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:22:59.962373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:22:59.962373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:22:59.962373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:22:59.962373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:22:59.962373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:22:59.962373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:22:59.962373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:22:59.962373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:22:59.962373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:22:59.962373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:22:59.962373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:22:59.962373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:22:59.962373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:22:59.962373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:22:59.962373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:22:59.963322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:22:59.963322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:22:59.963322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:22:59.963322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:22:59.963322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:22:59.963322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:22:59.963322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:22:59.963322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:22:59.963322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:22:59.963322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:22:59.963322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:22:59.963322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:22:59.963322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 578/1156] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:22:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:22:59.968988 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92ba73-f087-48ee-a69f-d8be8a49306a None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:00.065114 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92ba73-f087-48ee-a69f-d8be8a49306a None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:00.075146 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:00.101338 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03a457-ce90-40c4-9823-3519ac67493e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:00.102012 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03a457-ce90-40c4-9823-3519ac67493e None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:00.102492 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03a457-ce90-40c4-9823-3519ac67493e None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:00.102609 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03a457-ce90-40c4-9823-3519ac67493e None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:00.103323 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03a457-ce90-40c4-9823-3519ac67493e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:23:00.103628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03a457-ce90-40c4-9823-3519ac67493e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:00.104992 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03a457-ce90-40c4-9823-3519ac67493e None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:00.117848 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 579/1157] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:22:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:00.124301 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aadf1a9-ad77-402d-b78d-dddc605a4705 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:00.222889 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aadf1a9-ad77-402d-b78d-dddc605a4705 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:00.239596 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:00.257720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a2ad3-7ec9-4847-b0f9-2609c9760232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:00.258587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a2ad3-7ec9-4847-b0f9-2609c9760232 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:00.258871 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a2ad3-7ec9-4847-b0f9-2609c9760232 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:00.259148 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a2ad3-7ec9-4847-b0f9-2609c9760232 None admin] PATH_INFO: `/v3/users/e9d194a6cf404ad8b31e3768f2961f64` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:00.260323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a2ad3-7ec9-4847-b0f9-2609c9760232 None admin] RBAC: Authorizing `identity:get_user(user_id=e9d194a6cf404ad8b31e3768f2961f64)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:00.262384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a2ad3-7ec9-4847-b0f9-2609c9760232 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:00.264007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 579/1158] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:23:00 2022] GET /identity/v3/users/e9d194a6cf404ad8b31e3768f2961f64 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:00.269759 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03a457-ce90-40c4-9823-3519ac67493e None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:00.367728 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03a457-ce90-40c4-9823-3519ac67493e None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:00.401163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6ca01-9b35-4a1a-bd25-d087cf143abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:00.402021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6ca01-9b35-4a1a-bd25-d087cf143abb None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:00.402557 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6ca01-9b35-4a1a-bd25-d087cf143abb None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:00.402557 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6ca01-9b35-4a1a-bd25-d087cf143abb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:00.444100 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:00.483814 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6ca01-9b35-4a1a-bd25-d087cf143abb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:00.485543 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6ca01-9b35-4a1a-bd25-d087cf143abb None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:00.605899 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 580/1159] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:23:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:23:00.611798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a2ad3-7ec9-4847-b0f9-2609c9760232 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:00.708956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a2ad3-7ec9-4847-b0f9-2609c9760232 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:00.745585 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051766bd-1023-4da9-a64e-c4a857e5ff32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:00.746392 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051766bd-1023-4da9-a64e-c4a857e5ff32 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:00.746681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051766bd-1023-4da9-a64e-c4a857e5ff32 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:00.746963 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051766bd-1023-4da9-a64e-c4a857e5ff32 None admin] PATH_INFO: `/v3/projects/28304d8551614c85a8e45979874cbfcc` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:00.748145 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051766bd-1023-4da9-a64e-c4a857e5ff32 None admin] RBAC: Authorizing `identity:get_project(project_id=28304d8551614c85a8e45979874cbfcc)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:00.749660 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051766bd-1023-4da9-a64e-c4a857e5ff32 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:00.751369 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 580/1160] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:23:00 2022] GET /identity/v3/projects/28304d8551614c85a8e45979874cbfcc => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:00.757918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6ca01-9b35-4a1a-bd25-d087cf143abb None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:00.825159 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:00.855296 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6ca01-9b35-4a1a-bd25-d087cf143abb None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:00.892140 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db84972-031f-4198-9b54-d9406cee8587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:00.892809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db84972-031f-4198-9b54-d9406cee8587 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:00.893096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db84972-031f-4198-9b54-d9406cee8587 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:00.893404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db84972-031f-4198-9b54-d9406cee8587 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:00.894237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db84972-031f-4198-9b54-d9406cee8587 None admin] RBAC: Adding query filter params (role.id=25a16d8ea4084e9eacdebb7a4dcd2337, scope.project.id=28304d8551614c85a8e45979874cbfcc, user.id=e9d194a6cf404ad8b31e3768f2961f64) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:23:00.894554 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db84972-031f-4198-9b54-d9406cee8587 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:00.896027 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db84972-031f-4198-9b54-d9406cee8587 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:00.901740 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 581/1161] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:23:00 2022] GET /identity/v3/role_assignments?user.id=e9d194a6cf404ad8b31e3768f2961f64&scope.project.id=28304d8551614c85a8e45979874cbfcc&role.id=25a16d8ea4084e9eacdebb7a4dcd2337 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:03.028103 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53744a-ee68-4061-bdda-8f4bdacec324 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:03.028751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53744a-ee68-4061-bdda-8f4bdacec324 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:03.028751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53744a-ee68-4061-bdda-8f4bdacec324 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:03.030077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 581/1162] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:23:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:23:03.035272 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490696cf-680f-47f4-ac30-f00c12f396f1 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:03.035732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490696cf-680f-47f4-ac30-f00c12f396f1 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:03.035835 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490696cf-680f-47f4-ac30-f00c12f396f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:03.052660 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-490696cf-680f-47f4-ac30-f00c12f396f1 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:03.053884 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-490696cf-680f-47f4-ac30-f00c12f396f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:03.122433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 582/1163] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:23:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:23:03.146064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53744a-ee68-4061-bdda-8f4bdacec324 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:03.201494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:03.253224 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53744a-ee68-4061-bdda-8f4bdacec324 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:03.286565 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c0e34-9d04-43e1-b878-3f52d2cc1d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:03.287213 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c0e34-9d04-43e1-b878-3f52d2cc1d9e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:03.287499 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c0e34-9d04-43e1-b878-3f52d2cc1d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:03.287777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c0e34-9d04-43e1-b878-3f52d2cc1d9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:03.351756 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c0e34-9d04-43e1-b878-3f52d2cc1d9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:03.353226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c0e34-9d04-43e1-b878-3f52d2cc1d9e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:03.402272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:03.448217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 582/1164] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:23:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:23:03.454331 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490696cf-680f-47f4-ac30-f00c12f396f1 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:03.515357 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:03.552583 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490696cf-680f-47f4-ac30-f00c12f396f1 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:03.586166 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632c53b-86f2-433b-812d-12505c78d85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:03.586969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632c53b-86f2-433b-812d-12505c78d85f None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:03.587258 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632c53b-86f2-433b-812d-12505c78d85f None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:03.587541 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632c53b-86f2-433b-812d-12505c78d85f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:03.594292 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632c53b-86f2-433b-812d-12505c78d85f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:03.595731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632c53b-86f2-433b-812d-12505c78d85f None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:03.610378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-5632c53b-86f2-433b-812d-12505c78d85f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 17:23:03.610378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:23:03.610378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 16 17:23:03.610378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 16 17:23:03.610378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:23:03.610378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:23:03.610378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:23:03.610378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:23:03.610378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 16 17:23:03.610378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:23:03.610378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:23:03.610378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:23:03.610378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:23:03.610378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:23:03.610378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:23:03.610378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:23:03.610378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:23:03.611680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:23:03.611680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:23:03.611680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:23:03.611680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:23:03.611680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:23:03.611680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:23:03.611680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:23:03.611680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:23:03.611680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:23:03.611680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:23:03.611680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:23:03.611680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:23:03.611680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:23:03.611680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:23:03.611680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:23:03.611680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:23:03.612997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:23:03.612997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:23:03.612997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:23:03.612997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:23:03.612997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:23:03.612997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:23:03.612997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:23:03.612997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:23:03.612997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:23:03.612997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:23:03.612997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 16 17:23:03.612997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:23:03.612997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 17:23:03.612997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:23:03.612997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 583/1165] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jun 16 17:23:03 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:23:03.617618 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c0e34-9d04-43e1-b878-3f52d2cc1d9e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:03.717415 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c0e34-9d04-43e1-b878-3f52d2cc1d9e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:03.751120 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030182c4-dbda-4c1f-a874-8c79d4ce1a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:03.751780 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030182c4-dbda-4c1f-a874-8c79d4ce1a4f None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:03.752066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030182c4-dbda-4c1f-a874-8c79d4ce1a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:03.752344 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030182c4-dbda-4c1f-a874-8c79d4ce1a4f None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:03.753065 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030182c4-dbda-4c1f-a874-8c79d4ce1a4f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:23:03.753405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030182c4-dbda-4c1f-a874-8c79d4ce1a4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:03.754806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030182c4-dbda-4c1f-a874-8c79d4ce1a4f None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:03.762009 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 583/1166] 10.222.0.40 () {56 vars in 1115 bytes} [Thu Jun 16 17:23:03 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:05.920288 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05b1d2-44e4-48ce-84b9-08a6b6a89dd7 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:05.920852 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05b1d2-44e4-48ce-84b9-08a6b6a89dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:05.920852 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05b1d2-44e4-48ce-84b9-08a6b6a89dd7 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:05.922276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 584/1167] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:23:05.927483 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e64d70-2899-4f44-8f0f-5fb357ec9bd2 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:05.927925 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e64d70-2899-4f44-8f0f-5fb357ec9bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:05.928028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e64d70-2899-4f44-8f0f-5fb357ec9bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:05.945012 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-86e64d70-2899-4f44-8f0f-5fb357ec9bd2 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:05.946258 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-86e64d70-2899-4f44-8f0f-5fb357ec9bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:05.956979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:06.013122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 584/1168] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:23:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:23:06.036639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05b1d2-44e4-48ce-84b9-08a6b6a89dd7 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:06.138770 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05b1d2-44e4-48ce-84b9-08a6b6a89dd7 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:06.161787 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:06.173558 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221d243-b7ce-4178-b105-03d5787deecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:06.174165 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221d243-b7ce-4178-b105-03d5787deecc None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:06.174499 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221d243-b7ce-4178-b105-03d5787deecc None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:06.174789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221d243-b7ce-4178-b105-03d5787deecc None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:06.175556 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221d243-b7ce-4178-b105-03d5787deecc None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:06.176937 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221d243-b7ce-4178-b105-03d5787deecc None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:06.191415 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 585/1169] 10.222.0.40 () {60 vars in 1138 bytes} [Thu Jun 16 17:23:06 2022] POST /identity/v3/domains => generated 267 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:08.342373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8cf7c-7b78-4182-99ba-937912e88030 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:08.342996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8cf7c-7b78-4182-99ba-937912e88030 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:08.342996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8cf7c-7b78-4182-99ba-937912e88030 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:08.344331 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 585/1170] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:23:08.349708 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcff15-8449-4162-af2c-db2d3abead43 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:08.350111 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcff15-8449-4162-af2c-db2d3abead43 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:08.350286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcff15-8449-4162-af2c-db2d3abead43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:08.367659 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-84dcff15-8449-4162-af2c-db2d3abead43 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:08.368842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-84dcff15-8449-4162-af2c-db2d3abead43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:08.434972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 586/1171] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:23:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:23:08.458657 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8cf7c-7b78-4182-99ba-937912e88030 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:08.559548 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8cf7c-7b78-4182-99ba-937912e88030 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:08.593012 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe067813-1614-4ddf-b21e-e8075a7826c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:08.593898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe067813-1614-4ddf-b21e-e8075a7826c7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:08.594189 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe067813-1614-4ddf-b21e-e8075a7826c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:08.594475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe067813-1614-4ddf-b21e-e8075a7826c7 None admin] PATH_INFO: `/v3/domains/cfa79ebb0ba84a8d8e67ce75c5017778` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:08.605278 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe067813-1614-4ddf-b21e-e8075a7826c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cfa79ebb0ba84a8d8e67ce75c5017778)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:08.606722 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe067813-1614-4ddf-b21e-e8075a7826c7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:08.608365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 586/1172] 10.222.0.40 () {56 vars in 1183 bytes} [Thu Jun 16 17:23:08 2022] GET /identity/v3/domains/cfa79ebb0ba84a8d8e67ce75c5017778 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:08.614352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcff15-8449-4162-af2c-db2d3abead43 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:08.714460 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcff15-8449-4162-af2c-db2d3abead43 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:08.728593 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:08.750578 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b601d4-3b02-49a9-a513-b55ac607a1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:08.751241 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b601d4-3b02-49a9-a513-b55ac607a1db None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:08.751535 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b601d4-3b02-49a9-a513-b55ac607a1db None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:08.751819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b601d4-3b02-49a9-a513-b55ac607a1db None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:08.752528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b601d4-3b02-49a9-a513-b55ac607a1db None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:08.754079 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b601d4-3b02-49a9-a513-b55ac607a1db None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:08.780500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 587/1173] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 17:23:08 2022] POST /identity/v3/projects => generated 361 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:10.908106 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf3811-6e00-4129-b64b-8923ed575d41 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:10.908689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf3811-6e00-4129-b64b-8923ed575d41 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:10.908689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf3811-6e00-4129-b64b-8923ed575d41 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:10.910051 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 587/1174] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:23:10.916288 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a134c1-4b45-410a-b519-b7c657ade799 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:10.916681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a134c1-4b45-410a-b519-b7c657ade799 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:10.916870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a134c1-4b45-410a-b519-b7c657ade799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:10.935417 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-35a134c1-4b45-410a-b519-b7c657ade799 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:10.936610 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-35a134c1-4b45-410a-b519-b7c657ade799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:11.004224 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 588/1175] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:23:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:23:11.029938 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf3811-6e00-4129-b64b-8923ed575d41 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:11.039231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:11.130576 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf3811-6e00-4129-b64b-8923ed575d41 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:11.162839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2dd62b-4c31-47a3-9de3-bda58da40ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:11.163633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2dd62b-4c31-47a3-9de3-bda58da40ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:11.163995 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2dd62b-4c31-47a3-9de3-bda58da40ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:11.164273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2dd62b-4c31-47a3-9de3-bda58da40ce7 None admin] PATH_INFO: `/v3/domains/cfa79ebb0ba84a8d8e67ce75c5017778` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:11.165484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2dd62b-4c31-47a3-9de3-bda58da40ce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cfa79ebb0ba84a8d8e67ce75c5017778)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:11.166921 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2dd62b-4c31-47a3-9de3-bda58da40ce7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:11.169268 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 588/1176] 10.222.0.40 () {56 vars in 1183 bytes} [Thu Jun 16 17:23:11 2022] GET /identity/v3/domains/cfa79ebb0ba84a8d8e67ce75c5017778 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:11.175053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a134c1-4b45-410a-b519-b7c657ade799 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:11.268167 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:11.274930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a134c1-4b45-410a-b519-b7c657ade799 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:11.309132 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694eec5-4c49-4542-9eb9-bcf8f1c597bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:11.309817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694eec5-4c49-4542-9eb9-bcf8f1c597bc None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:11.310103 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694eec5-4c49-4542-9eb9-bcf8f1c597bc None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:11.310381 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694eec5-4c49-4542-9eb9-bcf8f1c597bc None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:11.311095 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694eec5-4c49-4542-9eb9-bcf8f1c597bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:11.312525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694eec5-4c49-4542-9eb9-bcf8f1c597bc None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:11.331427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 589/1177] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 17:23:11 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:13.515089 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c260a-7535-4404-b8b5-8257fcf69c50 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:13.515606 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c260a-7535-4404-b8b5-8257fcf69c50 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:13.515606 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c260a-7535-4404-b8b5-8257fcf69c50 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:13.516933 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 589/1178] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:23:13.522189 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7b1fb-fae5-4312-a709-2870b5ae073f None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:13.522656 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7b1fb-fae5-4312-a709-2870b5ae073f None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:13.522752 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7b1fb-fae5-4312-a709-2870b5ae073f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:13.539970 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-d5d7b1fb-fae5-4312-a709-2870b5ae073f None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:13.541179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-d5d7b1fb-fae5-4312-a709-2870b5ae073f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:13.608207 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 590/1179] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:23:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:23:13.632108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c260a-7535-4404-b8b5-8257fcf69c50 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:13.706866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:13.732966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c260a-7535-4404-b8b5-8257fcf69c50 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:13.768139 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbabdce-53c3-4dfc-aceb-5f98d5461ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:13.768855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbabdce-53c3-4dfc-aceb-5f98d5461ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:13.769141 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbabdce-53c3-4dfc-aceb-5f98d5461ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:13.769448 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbabdce-53c3-4dfc-aceb-5f98d5461ce2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:13.779718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbabdce-53c3-4dfc-aceb-5f98d5461ce2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:23:13.787252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-adbabdce-53c3-4dfc-aceb-5f98d5461ce2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:23:13.787252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:23:13.787252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:23:13.787252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:23:13.787252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:23:13.787252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:23:13.787252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:23:13.787252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:23:13.787252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:23:13.787252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:23:13.787252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:23:13.787252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:23:13.787252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:23:13.787252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:23:13.787252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:23:13.787252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:23:13.788405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:23:13.788405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:23:13.788405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:23:13.788405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:23:13.788405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:23:13.788405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:23:13.788405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:23:13.788405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:23:13.788405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:23:13.788405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:23:13.788405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:23:13.788405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:23:13.788405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:23:13.788405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:23:13.788405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:23:13.788405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:23:13.789351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:23:13.789351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:23:13.789351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:23:13.789351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:23:13.789351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:23:13.789351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:23:13.789351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:23:13.789351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:23:13.789351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:23:13.789351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:23:13.789351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:23:13.789351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:23:13.789351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 590/1180] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:23:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:23:13.795131 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d7b1fb-fae5-4312-a709-2870b5ae073f None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:13.875851 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:13.895333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d7b1fb-fae5-4312-a709-2870b5ae073f None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:13.931376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daf01d-3c23-421b-a02f-731d08bfa42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:13.932051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daf01d-3c23-421b-a02f-731d08bfa42b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:13.932349 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daf01d-3c23-421b-a02f-731d08bfa42b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:13.932635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daf01d-3c23-421b-a02f-731d08bfa42b None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:13.934032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66daf01d-3c23-421b-a02f-731d08bfa42b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:23:13.934348 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66daf01d-3c23-421b-a02f-731d08bfa42b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:13.935728 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66daf01d-3c23-421b-a02f-731d08bfa42b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:13.943804 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 591/1181] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:23:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:13.951152 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbabdce-53c3-4dfc-aceb-5f98d5461ce2 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:14.046895 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbabdce-53c3-4dfc-aceb-5f98d5461ce2 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:14.079001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd3f62-74aa-44af-a0fd-30cc720c8c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:14.079863 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd3f62-74aa-44af-a0fd-30cc720c8c8b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:14.080150 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd3f62-74aa-44af-a0fd-30cc720c8c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:14.080427 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd3f62-74aa-44af-a0fd-30cc720c8c8b None admin] PATH_INFO: `/v3/users/3ee510027c42473c83a7225fa5ceaafa` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:14.082336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:14.094350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd3f62-74aa-44af-a0fd-30cc720c8c8b None admin] RBAC: Authorizing `identity:get_user(user_id=3ee510027c42473c83a7225fa5ceaafa)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:14.095791 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd3f62-74aa-44af-a0fd-30cc720c8c8b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:14.097481 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 591/1182] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:23:13 2022] GET /identity/v3/users/3ee510027c42473c83a7225fa5ceaafa => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:14.103456 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daf01d-3c23-421b-a02f-731d08bfa42b None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:14.202464 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daf01d-3c23-421b-a02f-731d08bfa42b None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:14.235308 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7f051-7392-4f7e-abc5-4fee18580762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:14.235967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7f051-7392-4f7e-abc5-4fee18580762 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:14.236258 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7f051-7392-4f7e-abc5-4fee18580762 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:14.236542 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7f051-7392-4f7e-abc5-4fee18580762 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:14.251909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:14.314794 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff7f051-7392-4f7e-abc5-4fee18580762 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:14.316273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff7f051-7392-4f7e-abc5-4fee18580762 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:14.420048 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 592/1183] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:23:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:23:14.426036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd3f62-74aa-44af-a0fd-30cc720c8c8b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:14.529749 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd3f62-74aa-44af-a0fd-30cc720c8c8b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:14.564177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea77198-d92e-45a8-83e6-ca9744e2911e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:14.565082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea77198-d92e-45a8-83e6-ca9744e2911e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:14.565396 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea77198-d92e-45a8-83e6-ca9744e2911e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:14.565681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea77198-d92e-45a8-83e6-ca9744e2911e None admin] PATH_INFO: `/v3/projects/12559d6d656f41d9b84cf1ba7327bf35` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:14.566865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea77198-d92e-45a8-83e6-ca9744e2911e None admin] RBAC: Authorizing `identity:get_project(project_id=12559d6d656f41d9b84cf1ba7327bf35)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:14.568294 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea77198-d92e-45a8-83e6-ca9744e2911e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:14.570036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 592/1184] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:23:14 2022] GET /identity/v3/projects/12559d6d656f41d9b84cf1ba7327bf35 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:14.576458 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7f051-7392-4f7e-abc5-4fee18580762 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:14.627461 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:14.681392 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7f051-7392-4f7e-abc5-4fee18580762 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:14.715625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac1119f-40b4-46da-84d9-3e9b071b6650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:14.716277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1119f-40b4-46da-84d9-3e9b071b6650 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:14.716568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1119f-40b4-46da-84d9-3e9b071b6650 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:14.716851 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1119f-40b4-46da-84d9-3e9b071b6650 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:14.717722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac1119f-40b4-46da-84d9-3e9b071b6650 None admin] RBAC: Adding query filter params (role.id=25a16d8ea4084e9eacdebb7a4dcd2337, scope.project.id=12559d6d656f41d9b84cf1ba7327bf35, user.id=3ee510027c42473c83a7225fa5ceaafa) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:23:14.718043 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac1119f-40b4-46da-84d9-3e9b071b6650 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:14.719468 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac1119f-40b4-46da-84d9-3e9b071b6650 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:14.724887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 593/1185] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:23:14 2022] GET /identity/v3/role_assignments?user.id=3ee510027c42473c83a7225fa5ceaafa&scope.project.id=12559d6d656f41d9b84cf1ba7327bf35&role.id=25a16d8ea4084e9eacdebb7a4dcd2337 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:16.816145 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc3d06-5740-471a-8b7c-debc7a02e315 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:16.816691 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc3d06-5740-471a-8b7c-debc7a02e315 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:16.816691 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc3d06-5740-471a-8b7c-debc7a02e315 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:16.818119 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 593/1186] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:23:16.823240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019ffd3-1121-4362-b4bf-95eb9291fc80 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:16.823648 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019ffd3-1121-4362-b4bf-95eb9291fc80 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:16.823865 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019ffd3-1121-4362-b4bf-95eb9291fc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:16.840979 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-4019ffd3-1121-4362-b4bf-95eb9291fc80 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:16.842214 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4019ffd3-1121-4362-b4bf-95eb9291fc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:16.908307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 594/1187] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:23:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:23:16.931039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc3d06-5740-471a-8b7c-debc7a02e315 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:16.989882 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:17.031731 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc3d06-5740-471a-8b7c-debc7a02e315 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:17.064211 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24d6d3-b98f-424a-8d6d-691aeb4b5bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:17.065008 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24d6d3-b98f-424a-8d6d-691aeb4b5bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:17.065328 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24d6d3-b98f-424a-8d6d-691aeb4b5bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:17.065615 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24d6d3-b98f-424a-8d6d-691aeb4b5bf4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:17.076889 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24d6d3-b98f-424a-8d6d-691aeb4b5bf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:23:17.084445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-9f24d6d3-b98f-424a-8d6d-691aeb4b5bf4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:23:17.084445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:23:17.084445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:23:17.084445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:23:17.084445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:23:17.084445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:23:17.084445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:23:17.084445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:23:17.084445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:23:17.084445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:23:17.084445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:23:17.084445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:23:17.084445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:23:17.084445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:23:17.084445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:23:17.084445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:23:17.085605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:23:17.085605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:23:17.085605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:23:17.085605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:23:17.085605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:23:17.085605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:23:17.085605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:23:17.085605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:23:17.085605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:23:17.085605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:23:17.085605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:23:17.085605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:23:17.085605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:23:17.085605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:23:17.085605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:23:17.085605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:23:17.086829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:23:17.086829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:23:17.086829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:23:17.086829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:23:17.086829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:23:17.086829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:23:17.086829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:23:17.086829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:23:17.086829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:23:17.086829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:23:17.086829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:23:17.086829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:23:17.086829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 594/1188] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:23:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:23:17.092413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019ffd3-1121-4362-b4bf-95eb9291fc80 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:17.130679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:17.189509 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019ffd3-1121-4362-b4bf-95eb9291fc80 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:17.223092 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8483e82d-98c2-40e5-824a-802622b8b691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:17.223750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483e82d-98c2-40e5-824a-802622b8b691 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:17.224037 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483e82d-98c2-40e5-824a-802622b8b691 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:17.224311 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483e82d-98c2-40e5-824a-802622b8b691 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:17.225053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8483e82d-98c2-40e5-824a-802622b8b691 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:23:17.225393 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8483e82d-98c2-40e5-824a-802622b8b691 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:17.226744 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8483e82d-98c2-40e5-824a-802622b8b691 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:17.235120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 595/1189] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:23:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:17.240614 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24d6d3-b98f-424a-8d6d-691aeb4b5bf4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:17.343609 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24d6d3-b98f-424a-8d6d-691aeb4b5bf4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:17.363392 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:17.378579 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a980da25-c830-4a16-ac00-9494061f8ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:17.379461 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980da25-c830-4a16-ac00-9494061f8ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:17.380036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980da25-c830-4a16-ac00-9494061f8ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:17.380343 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980da25-c830-4a16-ac00-9494061f8ab3 None admin] PATH_INFO: `/v3/users/3ee510027c42473c83a7225fa5ceaafa` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:17.381574 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a980da25-c830-4a16-ac00-9494061f8ab3 None admin] RBAC: Authorizing `identity:get_user(user_id=3ee510027c42473c83a7225fa5ceaafa)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:17.383016 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a980da25-c830-4a16-ac00-9494061f8ab3 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:17.384671 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 595/1190] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:23:17 2022] GET /identity/v3/users/3ee510027c42473c83a7225fa5ceaafa => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:17.391503 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8483e82d-98c2-40e5-824a-802622b8b691 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:17.488101 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:17.491251 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8483e82d-98c2-40e5-824a-802622b8b691 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:17.525686 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76efe607-26b9-4417-8628-397362774b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:17.526779 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efe607-26b9-4417-8628-397362774b3a None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:17.526779 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efe607-26b9-4417-8628-397362774b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:17.527130 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efe607-26b9-4417-8628-397362774b3a None admin] PATH_INFO: `/v3/projects/12559d6d656f41d9b84cf1ba7327bf35` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:17.528312 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76efe607-26b9-4417-8628-397362774b3a None admin] RBAC: Authorizing `identity:get_project(project_id=12559d6d656f41d9b84cf1ba7327bf35)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:17.529778 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76efe607-26b9-4417-8628-397362774b3a None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:17.531505 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 596/1191] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:23:17 2022] GET /identity/v3/projects/12559d6d656f41d9b84cf1ba7327bf35 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:17.537468 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a980da25-c830-4a16-ac00-9494061f8ab3 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:17.636909 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a980da25-c830-4a16-ac00-9494061f8ab3 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:17.673075 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eff242-7cfd-40d9-abd3-07dea2700bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:17.673999 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eff242-7cfd-40d9-abd3-07dea2700bba None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:17.674288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eff242-7cfd-40d9-abd3-07dea2700bba None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:17.674571 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eff242-7cfd-40d9-abd3-07dea2700bba None admin] PATH_INFO: `/v3/projects/12559d6d656f41d9b84cf1ba7327bf35/users/3ee510027c42473c83a7225fa5ceaafa/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:17.676527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eff242-7cfd-40d9-abd3-07dea2700bba None admin] RBAC: Authorizing `identity:create_grant(project_id=12559d6d656f41d9b84cf1ba7327bf35, user_id=3ee510027c42473c83a7225fa5ceaafa, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:17.677984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eff242-7cfd-40d9-abd3-07dea2700bba None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:17.691453 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 596/1192] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:23:17 2022] PUT /identity/v3/projects/12559d6d656f41d9b84cf1ba7327bf35/users/3ee510027c42473c83a7225fa5ceaafa/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:19.780343 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6265fb-fc77-43fc-a62b-c8bf238bbd58 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:19.780886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6265fb-fc77-43fc-a62b-c8bf238bbd58 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:19.780886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6265fb-fc77-43fc-a62b-c8bf238bbd58 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:19.782343 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 597/1193] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:23:19.787371 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a76dd3-2be4-45f9-a329-46277ba24468 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:19.787817 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a76dd3-2be4-45f9-a329-46277ba24468 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:19.787931 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a76dd3-2be4-45f9-a329-46277ba24468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:19.805261 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-27a76dd3-2be4-45f9-a329-46277ba24468 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:19.806449 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-27a76dd3-2be4-45f9-a329-46277ba24468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:19.852336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:19.876459 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 597/1194] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:23:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:23:19.901019 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6265fb-fc77-43fc-a62b-c8bf238bbd58 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:20.004731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6265fb-fc77-43fc-a62b-c8bf238bbd58 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:20.037363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418aef5-5f74-4674-b4dd-4766719e2b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:20.038091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418aef5-5f74-4674-b4dd-4766719e2b56 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:20.038382 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418aef5-5f74-4674-b4dd-4766719e2b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:20.038666 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418aef5-5f74-4674-b4dd-4766719e2b56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:20.049593 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418aef5-5f74-4674-b4dd-4766719e2b56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:23:20.057219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-f418aef5-5f74-4674-b4dd-4766719e2b56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:23:20.057219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:23:20.057219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:23:20.057219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:23:20.057219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:23:20.057219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:23:20.057219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:23:20.057219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:23:20.057219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:23:20.057219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:23:20.057219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:23:20.057219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:23:20.057219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:23:20.057219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:23:20.057219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:23:20.057219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:23:20.058995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:23:20.058995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:23:20.058995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:23:20.058995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:23:20.058995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:23:20.058995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:23:20.058995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:23:20.058995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:23:20.058995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:23:20.058995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:23:20.058995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:23:20.058995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:23:20.058995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:23:20.058995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:23:20.058995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:23:20.058995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:23:20.059928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:23:20.059928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:23:20.059928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:23:20.059928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:23:20.059928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:23:20.059928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:23:20.059928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:23:20.059928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:23:20.059928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:23:20.059928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:23:20.059928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:23:20.059928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:23:20.059928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 598/1195] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:23:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:23:20.065343 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a76dd3-2be4-45f9-a329-46277ba24468 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:20.161205 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a76dd3-2be4-45f9-a329-46277ba24468 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:20.197351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081df582-c846-4d22-97a4-71b8cf7a09a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:20.198163 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081df582-c846-4d22-97a4-71b8cf7a09a2 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:20.198453 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081df582-c846-4d22-97a4-71b8cf7a09a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:20.198722 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081df582-c846-4d22-97a4-71b8cf7a09a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:20.199475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081df582-c846-4d22-97a4-71b8cf7a09a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:23:20.199787 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081df582-c846-4d22-97a4-71b8cf7a09a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:20.201179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081df582-c846-4d22-97a4-71b8cf7a09a2 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:20.209337 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 598/1196] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:23:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:20.215779 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418aef5-5f74-4674-b4dd-4766719e2b56 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:20.245518 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:20.319141 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418aef5-5f74-4674-b4dd-4766719e2b56 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:20.363918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9321ab3-3fc1-44a0-bb78-6b4b8eec2f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:20.364831 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9321ab3-3fc1-44a0-bb78-6b4b8eec2f27 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:20.365213 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9321ab3-3fc1-44a0-bb78-6b4b8eec2f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:20.365643 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9321ab3-3fc1-44a0-bb78-6b4b8eec2f27 None admin] PATH_INFO: `/v3/users/3ee510027c42473c83a7225fa5ceaafa` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:20.366830 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9321ab3-3fc1-44a0-bb78-6b4b8eec2f27 None admin] RBAC: Authorizing `identity:get_user(user_id=3ee510027c42473c83a7225fa5ceaafa)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:20.368257 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9321ab3-3fc1-44a0-bb78-6b4b8eec2f27 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:20.370313 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 599/1197] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 17:23:20 2022] GET /identity/v3/users/3ee510027c42473c83a7225fa5ceaafa => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:20.377809 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081df582-c846-4d22-97a4-71b8cf7a09a2 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:20.422480 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:20.485395 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081df582-c846-4d22-97a4-71b8cf7a09a2 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:20.522429 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99983287-cc64-4d93-aa61-1ebc46f48ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:20.523078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99983287-cc64-4d93-aa61-1ebc46f48ece None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:20.523368 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99983287-cc64-4d93-aa61-1ebc46f48ece None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:20.523649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99983287-cc64-4d93-aa61-1ebc46f48ece None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:20.590003 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99983287-cc64-4d93-aa61-1ebc46f48ece None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:20.591378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99983287-cc64-4d93-aa61-1ebc46f48ece None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:20.629791 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:20.698574 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 599/1198] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:23:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:23:20.705062 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9321ab3-3fc1-44a0-bb78-6b4b8eec2f27 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:20.799954 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:20.806100 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9321ab3-3fc1-44a0-bb78-6b4b8eec2f27 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:20.841066 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fcedbc-7b2d-4992-bbc8-7bbb4ace126a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:20.841890 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcedbc-7b2d-4992-bbc8-7bbb4ace126a None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:20.842183 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcedbc-7b2d-4992-bbc8-7bbb4ace126a None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:20.842465 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcedbc-7b2d-4992-bbc8-7bbb4ace126a None admin] PATH_INFO: `/v3/projects/12559d6d656f41d9b84cf1ba7327bf35` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:20.843649 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fcedbc-7b2d-4992-bbc8-7bbb4ace126a None admin] RBAC: Authorizing `identity:get_project(project_id=12559d6d656f41d9b84cf1ba7327bf35)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:20.845075 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fcedbc-7b2d-4992-bbc8-7bbb4ace126a None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:20.846803 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 600/1199] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 17:23:20 2022] GET /identity/v3/projects/12559d6d656f41d9b84cf1ba7327bf35 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:20.852909 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99983287-cc64-4d93-aa61-1ebc46f48ece None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:20.950619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99983287-cc64-4d93-aa61-1ebc46f48ece None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:20.985761 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98bdc1e-560d-4879-81e7-18503bdbab2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:20.986731 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bdc1e-560d-4879-81e7-18503bdbab2e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:20.987167 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bdc1e-560d-4879-81e7-18503bdbab2e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:20.987622 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bdc1e-560d-4879-81e7-18503bdbab2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:20.989018 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98bdc1e-560d-4879-81e7-18503bdbab2e None admin] RBAC: Adding query filter params (role.id=25a16d8ea4084e9eacdebb7a4dcd2337, scope.project.id=12559d6d656f41d9b84cf1ba7327bf35, user.id=3ee510027c42473c83a7225fa5ceaafa) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:23:20.989518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98bdc1e-560d-4879-81e7-18503bdbab2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:20.991754 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98bdc1e-560d-4879-81e7-18503bdbab2e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:21.001112 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 600/1200] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:23:20 2022] GET /identity/v3/role_assignments?user.id=3ee510027c42473c83a7225fa5ceaafa&scope.project.id=12559d6d656f41d9b84cf1ba7327bf35&role.id=25a16d8ea4084e9eacdebb7a4dcd2337 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:30.836364 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=18 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:30.996799 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=98 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:18.570593 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908952c-0b5b-4a72-95bf-317985c22132 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:18.571240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908952c-0b5b-4a72-95bf-317985c22132 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:18.571240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908952c-0b5b-4a72-95bf-317985c22132 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:18.572807 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 601/1201] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:24:18.579123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f71982-cf9e-42e0-8c78-3b040f27069b None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:18.579609 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f71982-cf9e-42e0-8c78-3b040f27069b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:18.579711 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f71982-cf9e-42e0-8c78-3b040f27069b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:18.598785 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-07f71982-cf9e-42e0-8c78-3b040f27069b None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:18.600052 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-07f71982-cf9e-42e0-8c78-3b040f27069b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:18.695312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 601/1202] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:24:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:24:18.727815 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908952c-0b5b-4a72-95bf-317985c22132 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:18.875238 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908952c-0b5b-4a72-95bf-317985c22132 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:18.919349 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962aba5-aa68-4893-b7f6-cce54008b721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:18.920345 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962aba5-aa68-4893-b7f6-cce54008b721 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:18.920638 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962aba5-aa68-4893-b7f6-cce54008b721 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:18.920918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962aba5-aa68-4893-b7f6-cce54008b721 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:18.935020 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962aba5-aa68-4893-b7f6-cce54008b721 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:18.936662 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962aba5-aa68-4893-b7f6-cce54008b721 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:18.961186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-6962aba5-aa68-4893-b7f6-cce54008b721 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:18.961186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:18.961186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:18.961186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:18.961186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:18.961186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:18.961186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:18.961186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:18.961186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:18.961186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:18.961186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:18.961186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:18.961186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:24:18.961186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:24:18.961186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:24:18.962745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:24:18.962745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:18.962745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:18.962745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:18.962745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:18.962745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:18.962745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:18.962745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:18.962745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:18.962745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:18.962745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:18.962745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:18.962745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:18.962745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:18.962745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:18.962745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:18.962745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:18.964216 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:24:18.964216 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:24:18.964216 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:18.964216 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:24:18.964216 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 602/1203] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:18.968864 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f71982-cf9e-42e0-8c78-3b040f27069b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:19.098336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f71982-cf9e-42e0-8c78-3b040f27069b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:19.135615 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:19.141963 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc1269-4ad3-488e-9af5-6d666fccc9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:19.142828 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc1269-4ad3-488e-9af5-6d666fccc9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.143373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc1269-4ad3-488e-9af5-6d666fccc9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.143792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc1269-4ad3-488e-9af5-6d666fccc9d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.144866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc1269-4ad3-488e-9af5-6d666fccc9d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:19.145443 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc1269-4ad3-488e-9af5-6d666fccc9d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:19.146940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc1269-4ad3-488e-9af5-6d666fccc9d7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:19.160821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 602/1204] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:19.167859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962aba5-aa68-4893-b7f6-cce54008b721 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:19.237642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:19.305069 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962aba5-aa68-4893-b7f6-cce54008b721 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:19.346879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7c7cd-64ad-44dc-a020-c65a583da26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:19.347786 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7c7cd-64ad-44dc-a020-c65a583da26e None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.348118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7c7cd-64ad-44dc-a020-c65a583da26e None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.348494 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7c7cd-64ad-44dc-a020-c65a583da26e None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.349282 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7c7cd-64ad-44dc-a020-c65a583da26e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:19.351289 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7c7cd-64ad-44dc-a020-c65a583da26e None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:19.381917 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 603/1205] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 17:24:19 2022] POST /identity/v3/users => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:24:21.645923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b4184-c5bf-4fc4-82b4-2f00149a7d81 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:21.646513 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b4184-c5bf-4fc4-82b4-2f00149a7d81 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:21.646513 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b4184-c5bf-4fc4-82b4-2f00149a7d81 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:21.649039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 603/1206] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:24:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:24:21.656141 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb931fa-0bfc-4e60-a71f-ee55dff0a09e None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:21.656613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb931fa-0bfc-4e60-a71f-ee55dff0a09e None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:21.656798 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb931fa-0bfc-4e60-a71f-ee55dff0a09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:21.674148 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-3fb931fa-0bfc-4e60-a71f-ee55dff0a09e None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:21.675469 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-3fb931fa-0bfc-4e60-a71f-ee55dff0a09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:21.764056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 604/1207] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:24:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:24:21.788262 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b4184-c5bf-4fc4-82b4-2f00149a7d81 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:21.906481 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b4184-c5bf-4fc4-82b4-2f00149a7d81 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:21.952144 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f41ad-eede-49f9-9168-0b5f04953ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:21.953200 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f41ad-eede-49f9-9168-0b5f04953ddf None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:21.953942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f41ad-eede-49f9-9168-0b5f04953ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:21.953942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f41ad-eede-49f9-9168-0b5f04953ddf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:21.968503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4f41ad-eede-49f9-9168-0b5f04953ddf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:24:21.978061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-7c4f41ad-eede-49f9-9168-0b5f04953ddf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:24:21.978061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:21.978061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:21.978061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:21.978061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:21.978061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:21.978061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:21.978061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:21.978061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:21.978061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:21.978061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:21.978061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:21.978061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:24:21.978061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:24:21.978061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:24:21.978061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:24:21.979134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:24:21.979134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:24:21.979134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:24:21.979134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:24:21.979134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:21.979134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:21.979134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:21.979134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:21.979134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:21.979134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:21.979134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:21.979134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:21.979134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:21.979134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:21.979134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:21.979134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:21.985107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:21.985107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:21.985107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:21.985107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:21.985107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:24:21.985107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:24:21.985107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:24:21.985107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:24:21.985107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:24:21.985107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:24:21.985107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:24:21.985107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:21.985107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 604/1208] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:24:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:24:21.985921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb931fa-0bfc-4e60-a71f-ee55dff0a09e None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:22.035676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:22.114073 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb931fa-0bfc-4e60-a71f-ee55dff0a09e None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:22.161178 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf784a-0128-4878-9d99-02d90fdf2c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:22.161804 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf784a-0128-4878-9d99-02d90fdf2c6d None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:22.162100 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf784a-0128-4878-9d99-02d90fdf2c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:22.162411 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf784a-0128-4878-9d99-02d90fdf2c6d None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:22.163337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf784a-0128-4878-9d99-02d90fdf2c6d None admin] RBAC: Adding query filter params (name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:22.163647 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf784a-0128-4878-9d99-02d90fdf2c6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:22.165021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf784a-0128-4878-9d99-02d90fdf2c6d None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:22.174922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 605/1209] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 17:24:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:22.180550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f41ad-eede-49f9-9168-0b5f04953ddf None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:22.241101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:22.307751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f41ad-eede-49f9-9168-0b5f04953ddf None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:22.350018 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2afce8-a024-440f-85a5-730c75492b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:22.350972 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2afce8-a024-440f-85a5-730c75492b7b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:22.351434 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2afce8-a024-440f-85a5-730c75492b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:22.351883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2afce8-a024-440f-85a5-730c75492b7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:22.373082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2afce8-a024-440f-85a5-730c75492b7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:22.374768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2afce8-a024-440f-85a5-730c75492b7b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:22.396580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-eb2afce8-a024-440f-85a5-730c75492b7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:22.396580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:22.396580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:22.396580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:22.396580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:22.396580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:22.396580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:22.396580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:22.396580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:22.396580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:22.396580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:22.396580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:22.396580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:24:22.396580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:24:22.396580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:24:22.398314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:24:22.398314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:22.398314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:22.398314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:22.398314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:22.398314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:22.398314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:22.398314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:22.398314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:22.398314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:22.398314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:22.398314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:22.398314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:22.398314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:22.398314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:22.398314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:22.398314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:22.402246 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:24:22.402246 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:24:22.402246 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:22.402246 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:22.402246 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 605/1210] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:22.404357 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf784a-0128-4878-9d99-02d90fdf2c6d None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:22.534839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:22.539219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf784a-0128-4878-9d99-02d90fdf2c6d None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:22.578353 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e027c-d375-406a-ac19-61b383bb4403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:22.579084 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e027c-d375-406a-ac19-61b383bb4403 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:22.579361 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e027c-d375-406a-ac19-61b383bb4403 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:22.579645 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e027c-d375-406a-ac19-61b383bb4403 None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:22.580382 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e027c-d375-406a-ac19-61b383bb4403 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:22.580700 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e027c-d375-406a-ac19-61b383bb4403 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:22.582112 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e027c-d375-406a-ac19-61b383bb4403 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:22.594995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 606/1211] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:24:22.601120 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2afce8-a024-440f-85a5-730c75492b7b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:22.731890 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2afce8-a024-440f-85a5-730c75492b7b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:22.733808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:22.782212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee3cb3-90db-4e78-8f3f-005c33072534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:22.783484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee3cb3-90db-4e78-8f3f-005c33072534 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:22.783893 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee3cb3-90db-4e78-8f3f-005c33072534 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:22.784476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee3cb3-90db-4e78-8f3f-005c33072534 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:22.792961 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee3cb3-90db-4e78-8f3f-005c33072534 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:22.794528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee3cb3-90db-4e78-8f3f-005c33072534 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:22.814263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-e0ee3cb3-90db-4e78-8f3f-005c33072534 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:24:22.814263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:22.814263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:22.814263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:22.814263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:22.814263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:22.814263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:22.814263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:22.814263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:22.814263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:22.814263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:22.814263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:22.814263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:24:22.814263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:24:22.814263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:24:22.814263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:24:22.815584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:22.815584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:22.815584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:24:22.815584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:24:22.815584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:24:22.815584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:24:22.815584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:22.815584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:22.815584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:22.815584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:22.815584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:22.815584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:22.815584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:22.815584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:22.815584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:22.815584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:22.815584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:22.815584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:22.817266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:22.817266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:22.817266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:24:22.817266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:24:22.817266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:24:22.817266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:24:22.817266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:24:22.817266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:24:22.817266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:24:22.817266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:22.819776 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 606/1212] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:24:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:22.828573 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e027c-d375-406a-ac19-61b383bb4403 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:22.973510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e027c-d375-406a-ac19-61b383bb4403 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:23.016070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf7ab7-e9f7-47af-bd05-b9fa04a1c8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:23.017018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf7ab7-e9f7-47af-bd05-b9fa04a1c8be None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:23.017488 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf7ab7-e9f7-47af-bd05-b9fa04a1c8be None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:23.018899 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf7ab7-e9f7-47af-bd05-b9fa04a1c8be None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:23.019926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf7ab7-e9f7-47af-bd05-b9fa04a1c8be None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:23.020407 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf7ab7-e9f7-47af-bd05-b9fa04a1c8be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:23.023208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf7ab7-e9f7-47af-bd05-b9fa04a1c8be None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:23.044334 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 607/1213] 10.222.0.40 () {56 vars in 1143 bytes} [Thu Jun 16 17:24:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:23.049935 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee3cb3-90db-4e78-8f3f-005c33072534 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:23.177760 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee3cb3-90db-4e78-8f3f-005c33072534 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:23.220650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d13dd7-f489-4db7-81a4-8b3d7511c1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:23.222351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d13dd7-f489-4db7-81a4-8b3d7511c1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:23.222513 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d13dd7-f489-4db7-81a4-8b3d7511c1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:23.222925 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d13dd7-f489-4db7-81a4-8b3d7511c1c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:23.268191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:23.316899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d13dd7-f489-4db7-81a4-8b3d7511c1c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:23.318311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d13dd7-f489-4db7-81a4-8b3d7511c1c4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:23.431893 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 607/1214] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:24:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:24:23.438176 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf7ab7-e9f7-47af-bd05-b9fa04a1c8be None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:23.458650 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:23.573685 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf7ab7-e9f7-47af-bd05-b9fa04a1c8be None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:23.623270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28445c-e5ee-430e-9bdd-849484e0a764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:23.624424 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28445c-e5ee-430e-9bdd-849484e0a764 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:23.624717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28445c-e5ee-430e-9bdd-849484e0a764 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:23.625056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28445c-e5ee-430e-9bdd-849484e0a764 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:23.639508 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a28445c-e5ee-430e-9bdd-849484e0a764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:23.641191 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a28445c-e5ee-430e-9bdd-849484e0a764 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:23.662383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-4a28445c-e5ee-430e-9bdd-849484e0a764 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:23.662383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:23.662383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:23.662383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:23.662383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:23.662383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:23.662383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:23.662383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:23.662383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:23.662383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:23.662383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:23.662383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:23.662383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:24:23.662383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:24:23.662383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:24:23.664163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:24:23.664163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:23.664163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:23.664163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:23.664163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:23.664163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:23.664163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:23.664163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:23.664163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:23.664163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:23.664163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:23.664163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:23.664163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:23.664163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:23.664163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:23.664163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:23.664163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:23.665346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:24:23.665346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:24:23.665346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:23.665346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:24:23.669234 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 608/1215] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:23.670281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d13dd7-f489-4db7-81a4-8b3d7511c1c4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:23.740873 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:23.786862 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d13dd7-f489-4db7-81a4-8b3d7511c1c4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:23.827653 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab6f71-46fb-445a-92d5-a117345fea3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:23.828371 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab6f71-46fb-445a-92d5-a117345fea3e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:23.828725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab6f71-46fb-445a-92d5-a117345fea3e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:23.829115 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab6f71-46fb-445a-92d5-a117345fea3e None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:23.830097 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab6f71-46fb-445a-92d5-a117345fea3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:23.830465 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab6f71-46fb-445a-92d5-a117345fea3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:23.832045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab6f71-46fb-445a-92d5-a117345fea3e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:23.846054 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 608/1216] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:23.852449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28445c-e5ee-430e-9bdd-849484e0a764 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:23.940441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:23.985300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28445c-e5ee-430e-9bdd-849484e0a764 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:24.023695 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57386043-06b4-416f-b941-dbfd3259d997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:24.024497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57386043-06b4-416f-b941-dbfd3259d997 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:24.024781 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57386043-06b4-416f-b941-dbfd3259d997 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:24.025093 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57386043-06b4-416f-b941-dbfd3259d997 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:24.033209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57386043-06b4-416f-b941-dbfd3259d997 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:24.034644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57386043-06b4-416f-b941-dbfd3259d997 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:24.048758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-57386043-06b4-416f-b941-dbfd3259d997 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:24:24.048758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:24.048758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:24.048758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:24.048758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:24.048758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:24.048758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:24.048758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:24.048758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:24.048758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:24.048758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:24.048758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:24.048758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:24:24.048758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:24:24.048758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:24:24.049902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:24:24.049902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:24.049902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:24.049902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:24.049902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:24.049902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:24.049902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:24.049902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:24.049902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:24.049902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:24.049902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:24.049902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:24.049902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:24.049902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:24.049902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:24.049902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:24.049902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:24.051056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:24:24.051056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:24:24.051056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:24:24.051056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:24:24.051056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:24:24.051056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:24:24.051056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:24:24.051056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:24:24.053478 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 609/1217] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:24:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:24.059252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab6f71-46fb-445a-92d5-a117345fea3e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:24.168517 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab6f71-46fb-445a-92d5-a117345fea3e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:24.192598 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:24.207870 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cb6eeb-9294-4b15-a9b3-aa6f1db5e90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:24.208703 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb6eeb-9294-4b15-a9b3-aa6f1db5e90e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:24.209086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb6eeb-9294-4b15-a9b3-aa6f1db5e90e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:24.210183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb6eeb-9294-4b15-a9b3-aa6f1db5e90e None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:24.211185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cb6eeb-9294-4b15-a9b3-aa6f1db5e90e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:24.211595 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cb6eeb-9294-4b15-a9b3-aa6f1db5e90e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:24.213249 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cb6eeb-9294-4b15-a9b3-aa6f1db5e90e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:24.232596 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 609/1218] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:24:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:24.232900 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57386043-06b4-416f-b941-dbfd3259d997 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:24.351435 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57386043-06b4-416f-b941-dbfd3259d997 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:24.383965 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:24.390727 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2278200-afa8-4850-887c-5fef55dc11d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:24.391405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2278200-afa8-4850-887c-5fef55dc11d6 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:24.391752 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2278200-afa8-4850-887c-5fef55dc11d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:24.392014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2278200-afa8-4850-887c-5fef55dc11d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:24.392830 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2278200-afa8-4850-887c-5fef55dc11d6 None admin] RBAC: Adding query filter params (role.id=0e34a6f9b89a4e29951fb189abacb34a, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=75f0586a8c1941448d750918183f6dc4) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:24.393224 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2278200-afa8-4850-887c-5fef55dc11d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:24.394669 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2278200-afa8-4850-887c-5fef55dc11d6 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:24.402900 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 610/1219] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:24:24 2022] GET /identity/v3/role_assignments?user.id=75f0586a8c1941448d750918183f6dc4&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=0e34a6f9b89a4e29951fb189abacb34a => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:26.613817 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87521ce-257e-4e18-8cc5-307e68287fe9 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:26.614384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87521ce-257e-4e18-8cc5-307e68287fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:26.614384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87521ce-257e-4e18-8cc5-307e68287fe9 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:26.616310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 610/1220] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:24:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:24:26.622577 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116474e-c4ba-4039-9378-d68f1dec08c2 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:26.623016 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116474e-c4ba-4039-9378-d68f1dec08c2 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:26.623295 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116474e-c4ba-4039-9378-d68f1dec08c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:26.641325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-0116474e-c4ba-4039-9378-d68f1dec08c2 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:26.642548 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0116474e-c4ba-4039-9378-d68f1dec08c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:26.715918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 611/1221] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:24:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:24:26.741501 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87521ce-257e-4e18-8cc5-307e68287fe9 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:26.854718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87521ce-257e-4e18-8cc5-307e68287fe9 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:26.895559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ca1e9-500e-441b-9ec4-32700e5720f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:26.896340 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ca1e9-500e-441b-9ec4-32700e5720f3 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:26.897264 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ca1e9-500e-441b-9ec4-32700e5720f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:26.897570 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ca1e9-500e-441b-9ec4-32700e5720f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:26.909505 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ca1e9-500e-441b-9ec4-32700e5720f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:24:26.918215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-082ca1e9-500e-441b-9ec4-32700e5720f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:24:26.918215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:26.918215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:26.918215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:26.918215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:26.918215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:26.918215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:26.918215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:26.918215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:26.918215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:26.918215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:26.918215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:26.918215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:24:26.918215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:24:26.918215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:24:26.918215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:24:26.919382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:24:26.919382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:24:26.919382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:24:26.919382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:24:26.919382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:26.919382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:26.919382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:26.919382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:26.919382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:26.919382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:26.919382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:26.919382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:26.919382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:26.919382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:26.919382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:26.919382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:26.920439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:26.920439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:26.920439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:26.920439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:26.920439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:24:26.920439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:24:26.920439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:24:26.920439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:24:26.920439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:24:26.920439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:24:26.920439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:24:26.920439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:26.925224 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 611/1222] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:24:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:26.925609 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116474e-c4ba-4039-9378-d68f1dec08c2 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:27.033474 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116474e-c4ba-4039-9378-d68f1dec08c2 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:27.058715 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:27.070337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfc1d4-84cf-4b9c-935e-77e6aec88940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:27.071020 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfc1d4-84cf-4b9c-935e-77e6aec88940 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:27.071309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfc1d4-84cf-4b9c-935e-77e6aec88940 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:27.071590 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfc1d4-84cf-4b9c-935e-77e6aec88940 None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:27.072357 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebfc1d4-84cf-4b9c-935e-77e6aec88940 None admin] RBAC: Adding query filter params (name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:27.072670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebfc1d4-84cf-4b9c-935e-77e6aec88940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:27.074072 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebfc1d4-84cf-4b9c-935e-77e6aec88940 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:27.082403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 612/1223] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 17:24:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:24:27.088550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ca1e9-500e-441b-9ec4-32700e5720f3 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:27.134888 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:27.196474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ca1e9-500e-441b-9ec4-32700e5720f3 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:27.237782 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e6306-edbc-4db2-ada0-3bf54343c849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:27.238679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e6306-edbc-4db2-ada0-3bf54343c849 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:27.239040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e6306-edbc-4db2-ada0-3bf54343c849 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:27.239383 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e6306-edbc-4db2-ada0-3bf54343c849 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:27.252864 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e6306-edbc-4db2-ada0-3bf54343c849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:27.254450 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e6306-edbc-4db2-ada0-3bf54343c849 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:27.275193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-8c9e6306-edbc-4db2-ada0-3bf54343c849 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:27.275193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:27.275193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:27.275193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:27.275193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:27.275193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:27.275193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:27.275193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:27.275193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:27.275193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:27.275193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:27.275193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:27.275193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:24:27.275193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:24:27.275193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:24:27.276218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:24:27.276218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:27.276218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:27.276218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:27.276218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:27.276218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:27.276218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:27.276218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:27.276218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:27.276218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:27.276218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:27.276218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:27.276218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:27.276218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:27.276218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:27.276218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:27.276218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:27.277183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:24:27.277183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:24:27.277183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:27.277183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:27.277183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 612/1224] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:27.283126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfc1d4-84cf-4b9c-935e-77e6aec88940 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:27.391869 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfc1d4-84cf-4b9c-935e-77e6aec88940 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:27.429195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e3d26-7d9e-4235-973a-24069f8cc43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:27.429195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e3d26-7d9e-4235-973a-24069f8cc43f None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:27.429195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e3d26-7d9e-4235-973a-24069f8cc43f None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:27.429195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e3d26-7d9e-4235-973a-24069f8cc43f None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:27.429903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e3d26-7d9e-4235-973a-24069f8cc43f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:27.430228 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e3d26-7d9e-4235-973a-24069f8cc43f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:27.431858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e3d26-7d9e-4235-973a-24069f8cc43f None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:27.446861 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 613/1225] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:27.452903 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e6306-edbc-4db2-ada0-3bf54343c849 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:27.561480 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:27.569663 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e6306-edbc-4db2-ada0-3bf54343c849 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:27.608432 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9933154-1c09-47bf-8832-5cb45c46eb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:27.609668 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9933154-1c09-47bf-8832-5cb45c46eb67 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:27.610228 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9933154-1c09-47bf-8832-5cb45c46eb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:27.610511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9933154-1c09-47bf-8832-5cb45c46eb67 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:27.621114 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9933154-1c09-47bf-8832-5cb45c46eb67 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:27.622778 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9933154-1c09-47bf-8832-5cb45c46eb67 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:27.641567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-b9933154-1c09-47bf-8832-5cb45c46eb67 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:24:27.641567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:27.641567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:27.641567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:27.641567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:27.641567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:27.641567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:27.641567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:27.641567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:27.641567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:27.641567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:27.641567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:27.641567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:24:27.641567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:24:27.641567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:24:27.641567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:24:27.642923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:27.642923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:27.642923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:24:27.642923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:24:27.642923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:24:27.642923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:24:27.642923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:27.642923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:27.642923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:27.642923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:27.642923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:27.642923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:27.642923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:27.642923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:27.642923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:27.642923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:27.642923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:27.642923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:27.644497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:27.644497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:27.644497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:24:27.644497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:24:27.644497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:24:27.644497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:24:27.644497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:24:27.644497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:24:27.644497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:24:27.644497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:27.644497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 613/1226] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:24:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:27.650135 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e3d26-7d9e-4235-973a-24069f8cc43f None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:27.714048 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:27.775536 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e3d26-7d9e-4235-973a-24069f8cc43f None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:27.820470 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb53d86f-8c71-4e73-975d-435fc37395ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:27.821261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53d86f-8c71-4e73-975d-435fc37395ab None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:27.821678 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53d86f-8c71-4e73-975d-435fc37395ab None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:27.822000 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53d86f-8c71-4e73-975d-435fc37395ab None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:27.823070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb53d86f-8c71-4e73-975d-435fc37395ab None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:27.823428 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb53d86f-8c71-4e73-975d-435fc37395ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:27.825099 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb53d86f-8c71-4e73-975d-435fc37395ab None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:27.847290 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 614/1227] 10.222.0.40 () {56 vars in 1143 bytes} [Thu Jun 16 17:24:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:27.853696 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9933154-1c09-47bf-8832-5cb45c46eb67 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:27.991641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9933154-1c09-47bf-8832-5cb45c46eb67 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:28.041442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d1e9f-f6b5-4175-97ea-0946a5bfe208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:28.042581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d1e9f-f6b5-4175-97ea-0946a5bfe208 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:28.042873 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d1e9f-f6b5-4175-97ea-0946a5bfe208 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:28.043154 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d1e9f-f6b5-4175-97ea-0946a5bfe208 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:28.060042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d1e9f-f6b5-4175-97ea-0946a5bfe208 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:28.061765 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d1e9f-f6b5-4175-97ea-0946a5bfe208 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:28.083419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-4b7d1e9f-f6b5-4175-97ea-0946a5bfe208 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:28.083419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:28.083419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:28.083419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:28.083419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:28.083419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:28.083419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:28.083419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:28.083419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:28.083419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:28.083419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:28.083419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:28.083419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:24:28.083419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:24:28.083419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:24:28.086077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:24:28.086077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:28.086077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:28.086077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:28.086077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:28.086077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:28.086077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:28.086077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:28.086077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:28.086077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:28.086077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:28.086077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:28.086077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:28.086077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:28.086077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:28.086077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:28.086077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:28.087903 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:24:28.087903 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:24:28.087903 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:28.087903 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:28.087903 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 614/1228] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:24:28.092368 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb53d86f-8c71-4e73-975d-435fc37395ab None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:28.238129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb53d86f-8c71-4e73-975d-435fc37395ab None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:28.241438 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:28.289322 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b4bea-39e1-4d5e-818f-c924187c4ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:28.290485 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b4bea-39e1-4d5e-818f-c924187c4ebf None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:28.291068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b4bea-39e1-4d5e-818f-c924187c4ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:28.291396 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b4bea-39e1-4d5e-818f-c924187c4ebf None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:28.292775 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b4bea-39e1-4d5e-818f-c924187c4ebf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:28.293173 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b4bea-39e1-4d5e-818f-c924187c4ebf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:28.295957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b4bea-39e1-4d5e-818f-c924187c4ebf None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:28.314732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 615/1229] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:28.321074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d1e9f-f6b5-4175-97ea-0946a5bfe208 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:28.331574 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:28.448492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d1e9f-f6b5-4175-97ea-0946a5bfe208 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:28.495176 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962275cd-7348-4046-aa8f-94463505f669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:28.496246 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962275cd-7348-4046-aa8f-94463505f669 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:28.496672 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962275cd-7348-4046-aa8f-94463505f669 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:28.497209 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962275cd-7348-4046-aa8f-94463505f669 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:28.510567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962275cd-7348-4046-aa8f-94463505f669 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:28.512338 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962275cd-7348-4046-aa8f-94463505f669 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:28.529222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-962275cd-7348-4046-aa8f-94463505f669 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:24:28.529222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:28.529222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:28.529222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:28.529222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:28.529222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:28.529222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:28.529222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:28.529222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:28.529222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:28.529222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:28.529222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:28.529222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:24:28.529222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:24:28.529222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:24:28.530433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:24:28.530433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:28.530433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:28.530433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:28.530433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:28.530433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:28.530433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:28.530433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:28.530433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:28.530433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:28.530433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:28.530433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:28.530433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:28.530433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:28.530433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:28.530433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:28.530433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:28.538182 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:24:28.538182 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:24:28.538182 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:24:28.538182 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:24:28.538182 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:24:28.538182 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:24:28.538182 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:24:28.538182 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:28.538182 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 615/1230] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:24:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:28.540230 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b4bea-39e1-4d5e-818f-c924187c4ebf None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:28.675458 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b4bea-39e1-4d5e-818f-c924187c4ebf None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:28.721201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb19b2e-1f11-403e-bf99-ecbae6e284b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:28.721874 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb19b2e-1f11-403e-bf99-ecbae6e284b5 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:28.722164 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb19b2e-1f11-403e-bf99-ecbae6e284b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:28.722444 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb19b2e-1f11-403e-bf99-ecbae6e284b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:28.723246 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb19b2e-1f11-403e-bf99-ecbae6e284b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:28.723559 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb19b2e-1f11-403e-bf99-ecbae6e284b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:28.725327 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb19b2e-1f11-403e-bf99-ecbae6e284b5 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:28.738912 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 616/1231] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:24:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:28.745831 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962275cd-7348-4046-aa8f-94463505f669 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:28.826732 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:28.864193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962275cd-7348-4046-aa8f-94463505f669 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:28.908694 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06fa119-6d83-41e3-959d-59b8a22fb55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:28.909770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06fa119-6d83-41e3-959d-59b8a22fb55a None admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:28.910099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06fa119-6d83-41e3-959d-59b8a22fb55a None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:28.910499 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06fa119-6d83-41e3-959d-59b8a22fb55a None admin] PATH_INFO: `/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/75f0586a8c1941448d750918183f6dc4/roles/0e34a6f9b89a4e29951fb189abacb34a` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:28.929433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06fa119-6d83-41e3-959d-59b8a22fb55a None admin] RBAC: Authorizing `identity:create_grant(project_id=35a535a305e740368b7a4d77d0ad22a0, user_id=75f0586a8c1941448d750918183f6dc4, role_id=0e34a6f9b89a4e29951fb189abacb34a)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:28.930895 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06fa119-6d83-41e3-959d-59b8a22fb55a None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:28.943842 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 616/1232] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:24:28 2022] PUT /identity/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/75f0586a8c1941448d750918183f6dc4/roles/0e34a6f9b89a4e29951fb189abacb34a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:31.183520 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadee9b3-2452-4b29-a970-6b1c9f6121f3 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:31.184076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadee9b3-2452-4b29-a970-6b1c9f6121f3 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:31.184076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadee9b3-2452-4b29-a970-6b1c9f6121f3 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:31.185686 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 617/1233] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:24:31.190643 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06a4e8-fe7d-42af-beee-349cd1300af4 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:31.191085 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06a4e8-fe7d-42af-beee-349cd1300af4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:31.191395 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06a4e8-fe7d-42af-beee-349cd1300af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:31.211942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-2e06a4e8-fe7d-42af-beee-349cd1300af4 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:31.213267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2e06a4e8-fe7d-42af-beee-349cd1300af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:31.294769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 617/1234] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:24:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:24:31.322303 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadee9b3-2452-4b29-a970-6b1c9f6121f3 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:31.351390 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:31.439946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadee9b3-2452-4b29-a970-6b1c9f6121f3 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:31.486403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a9789-0c61-49fc-a4c5-3d2fc4556351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:31.487448 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a9789-0c61-49fc-a4c5-3d2fc4556351 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:31.487763 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a9789-0c61-49fc-a4c5-3d2fc4556351 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:31.488043 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a9789-0c61-49fc-a4c5-3d2fc4556351 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:31.500232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9a9789-0c61-49fc-a4c5-3d2fc4556351 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:24:31.508758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-4e9a9789-0c61-49fc-a4c5-3d2fc4556351 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:24:31.508758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:31.508758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:31.508758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:31.508758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:31.508758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:31.508758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:31.508758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:31.508758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:31.508758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:31.508758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:31.508758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:31.508758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:24:31.508758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:24:31.508758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:24:31.508758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:24:31.509993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:24:31.509993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:24:31.509993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:24:31.509993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:24:31.509993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:31.509993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:31.509993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:31.509993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:31.509993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:31.509993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:31.509993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:31.509993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:31.509993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:31.509993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:31.509993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:31.509993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:31.511300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:31.511300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:31.511300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:31.511300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:31.511300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:24:31.511300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:24:31.511300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:24:31.511300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:24:31.511300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:24:31.511300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:24:31.511300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:24:31.511300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:24:31.511300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 618/1235] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 17:24:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:31.516245 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06a4e8-fe7d-42af-beee-349cd1300af4 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:31.575479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:31.634673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06a4e8-fe7d-42af-beee-349cd1300af4 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:31.670766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74f47e-42d0-46c5-9d03-3e2874b4e527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:31.671485 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74f47e-42d0-46c5-9d03-3e2874b4e527 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:31.671777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74f47e-42d0-46c5-9d03-3e2874b4e527 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:31.672061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74f47e-42d0-46c5-9d03-3e2874b4e527 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:31.672819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74f47e-42d0-46c5-9d03-3e2874b4e527 None admin] RBAC: Adding query filter params (name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:31.673132 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74f47e-42d0-46c5-9d03-3e2874b4e527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:31.674657 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74f47e-42d0-46c5-9d03-3e2874b4e527 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:31.684622 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 618/1236] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 17:24:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:31.690680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a9789-0c61-49fc-a4c5-3d2fc4556351 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:31.793483 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:31.812586 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a9789-0c61-49fc-a4c5-3d2fc4556351 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:31.849839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa3282-e752-4fec-96de-900ec8b9083c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:31.850656 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa3282-e752-4fec-96de-900ec8b9083c None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:31.850954 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa3282-e752-4fec-96de-900ec8b9083c None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:31.851234 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa3282-e752-4fec-96de-900ec8b9083c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:31.862829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa3282-e752-4fec-96de-900ec8b9083c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:31.864403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa3282-e752-4fec-96de-900ec8b9083c None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:31.888197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-80fa3282-e752-4fec-96de-900ec8b9083c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:31.888197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:31.888197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:31.888197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:31.888197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:31.888197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:31.888197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:31.888197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:31.888197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:31.888197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:31.888197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:31.888197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:31.888197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:24:31.888197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:24:31.888197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:24:31.889688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:24:31.889688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:31.889688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:31.889688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:31.889688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:31.889688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:31.889688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:31.889688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:31.889688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:31.889688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:31.889688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:31.889688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:31.889688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:31.889688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:31.889688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:31.889688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:31.889688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:31.890630 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:24:31.890630 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:24:31.890630 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:31.890630 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:24:31.890630 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 619/1237] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:31.895746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74f47e-42d0-46c5-9d03-3e2874b4e527 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:32.011818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74f47e-42d0-46c5-9d03-3e2874b4e527 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:32.022534 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:32.047674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f671164-69fa-43b6-ab92-3617528475b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:32.048472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f671164-69fa-43b6-ab92-3617528475b5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:32.048845 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f671164-69fa-43b6-ab92-3617528475b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:32.049136 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f671164-69fa-43b6-ab92-3617528475b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:32.049978 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f671164-69fa-43b6-ab92-3617528475b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:32.050293 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f671164-69fa-43b6-ab92-3617528475b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:32.051843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f671164-69fa-43b6-ab92-3617528475b5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:32.064807 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 619/1238] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:32.071384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa3282-e752-4fec-96de-900ec8b9083c None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:32.184588 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa3282-e752-4fec-96de-900ec8b9083c None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:32.220945 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dceca0f-d151-460f-8fc6-f971351d503f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:32.221771 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dceca0f-d151-460f-8fc6-f971351d503f None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:32.222060 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dceca0f-d151-460f-8fc6-f971351d503f None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:32.222344 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dceca0f-d151-460f-8fc6-f971351d503f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:32.225693 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:32.232596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dceca0f-d151-460f-8fc6-f971351d503f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:32.234090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dceca0f-d151-460f-8fc6-f971351d503f None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:32.249789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-1dceca0f-d151-460f-8fc6-f971351d503f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:24:32.249789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:32.249789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:32.249789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:32.249789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:32.249789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:32.249789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:32.249789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:32.249789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:32.249789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:32.249789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:32.249789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:32.249789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:24:32.249789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:24:32.249789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:24:32.249789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:24:32.251346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:32.251346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:32.251346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:24:32.251346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:24:32.251346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:24:32.251346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:24:32.251346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:32.251346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:32.251346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:32.251346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:32.251346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:32.251346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:32.251346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:32.251346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:32.251346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:32.251346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:32.251346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:32.251346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:32.252523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:32.252523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:32.252523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:24:32.252523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:24:32.252523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:24:32.252523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:24:32.252523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:24:32.252523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:24:32.252523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:24:32.252523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:24:32.256696 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 620/1239] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:24:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:32.257605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f671164-69fa-43b6-ab92-3617528475b5 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:32.371313 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f671164-69fa-43b6-ab92-3617528475b5 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:32.413064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5caa1c9-bebb-4951-8d00-dc0cf261b81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:32.413771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5caa1c9-bebb-4951-8d00-dc0cf261b81b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:32.414056 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5caa1c9-bebb-4951-8d00-dc0cf261b81b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:32.414337 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5caa1c9-bebb-4951-8d00-dc0cf261b81b None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:32.415178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5caa1c9-bebb-4951-8d00-dc0cf261b81b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:32.415490 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5caa1c9-bebb-4951-8d00-dc0cf261b81b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:32.416910 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5caa1c9-bebb-4951-8d00-dc0cf261b81b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:32.438394 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 620/1240] 10.222.0.40 () {56 vars in 1143 bytes} [Thu Jun 16 17:24:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:32.444437 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dceca0f-d151-460f-8fc6-f971351d503f None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:32.563338 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dceca0f-d151-460f-8fc6-f971351d503f None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:32.599862 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b911b-0f21-4925-8324-1a71beca99c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:32.600608 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b911b-0f21-4925-8324-1a71beca99c9 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:32.601004 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b911b-0f21-4925-8324-1a71beca99c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:32.601309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b911b-0f21-4925-8324-1a71beca99c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:32.673529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:32.680990 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b911b-0f21-4925-8324-1a71beca99c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:32.682670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b911b-0f21-4925-8324-1a71beca99c9 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:32.798679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 621/1241] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:24:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:24:32.805373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5caa1c9-bebb-4951-8d00-dc0cf261b81b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:32.832957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:32.920850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5caa1c9-bebb-4951-8d00-dc0cf261b81b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:32.961220 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1c01b-4064-4b8d-8226-9f242c6cc283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:32.961953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1c01b-4064-4b8d-8226-9f242c6cc283 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:32.962239 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1c01b-4064-4b8d-8226-9f242c6cc283 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:32.962605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1c01b-4064-4b8d-8226-9f242c6cc283 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:32.977811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1c01b-4064-4b8d-8226-9f242c6cc283 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:32.980042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1c01b-4064-4b8d-8226-9f242c6cc283 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:33.002380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-55f1c01b-4064-4b8d-8226-9f242c6cc283 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:33.002380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:33.002380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:33.002380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:33.002380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:33.002380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:33.002380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:33.002380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:33.002380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:33.002380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:33.002380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:33.002380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:33.002380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:24:33.002380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:24:33.002380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:24:33.003556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:24:33.003556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:33.003556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:33.003556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:33.003556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:33.003556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:33.003556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:33.003556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:33.003556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:33.003556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:33.003556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:33.003556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:33.003556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:33.003556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:33.003556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:33.003556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:33.003556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:33.009885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:24:33.009885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:24:33.009885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:33.009885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:33.009885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 621/1242] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:33.010373 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b911b-0f21-4925-8324-1a71beca99c9 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:33.121396 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:33.128904 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b911b-0f21-4925-8324-1a71beca99c9 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:33.172103 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9f35c-ffe1-4077-b722-1e5353baab7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:33.172775 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9f35c-ffe1-4077-b722-1e5353baab7f None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:33.173063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9f35c-ffe1-4077-b722-1e5353baab7f None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:33.173372 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9f35c-ffe1-4077-b722-1e5353baab7f None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:33.174102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9f35c-ffe1-4077-b722-1e5353baab7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:33.174413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9f35c-ffe1-4077-b722-1e5353baab7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:33.176033 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9f35c-ffe1-4077-b722-1e5353baab7f None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:33.189647 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 622/1243] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:33.196184 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1c01b-4064-4b8d-8226-9f242c6cc283 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:33.287852 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:33.314985 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1c01b-4064-4b8d-8226-9f242c6cc283 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:33.354642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67b5dd-9dc1-45ae-b418-b3895dabc3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:33.355458 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67b5dd-9dc1-45ae-b418-b3895dabc3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:33.355746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67b5dd-9dc1-45ae-b418-b3895dabc3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:33.356017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67b5dd-9dc1-45ae-b418-b3895dabc3c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:33.363849 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67b5dd-9dc1-45ae-b418-b3895dabc3c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:33.366118 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67b5dd-9dc1-45ae-b418-b3895dabc3c7 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:33.382841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-ca67b5dd-9dc1-45ae-b418-b3895dabc3c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:24:33.382841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:33.382841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:33.382841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:33.382841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:33.382841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:33.382841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:33.382841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:33.382841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:33.382841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:33.382841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:33.382841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:33.382841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:24:33.382841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:24:33.382841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:24:33.384040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:24:33.384040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:33.384040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:33.384040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:33.384040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:33.384040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:33.384040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:33.384040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:33.384040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:33.384040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:33.384040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:33.384040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:33.384040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:33.384040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:33.384040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:33.384040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:33.384040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:33.385000 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:24:33.385000 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:24:33.385000 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:24:33.385000 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:24:33.385000 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:24:33.385000 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:24:33.385000 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:24:33.385000 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:33.385000 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 622/1244] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:24:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:33.390013 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9f35c-ffe1-4077-b722-1e5353baab7f None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:33.499546 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9f35c-ffe1-4077-b722-1e5353baab7f None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:33.540371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699bed3-86e1-44e7-ad03-de56e6fd45e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:33.541142 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699bed3-86e1-44e7-ad03-de56e6fd45e6 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:33.541533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699bed3-86e1-44e7-ad03-de56e6fd45e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:33.541894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699bed3-86e1-44e7-ad03-de56e6fd45e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:33.542769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6699bed3-86e1-44e7-ad03-de56e6fd45e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:33.543163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6699bed3-86e1-44e7-ad03-de56e6fd45e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:33.544675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6699bed3-86e1-44e7-ad03-de56e6fd45e6 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:33.557112 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 623/1245] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:24:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:33.564170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67b5dd-9dc1-45ae-b418-b3895dabc3c7 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:33.672562 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67b5dd-9dc1-45ae-b418-b3895dabc3c7 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:33.708737 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac05165-55e6-4339-84f2-7a193ec45ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:33.709338 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac05165-55e6-4339-84f2-7a193ec45ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:33.709631 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac05165-55e6-4339-84f2-7a193ec45ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:33.709919 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac05165-55e6-4339-84f2-7a193ec45ab6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:33.710762 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac05165-55e6-4339-84f2-7a193ec45ab6 None admin] RBAC: Adding query filter params (role.id=0e34a6f9b89a4e29951fb189abacb34a, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=75f0586a8c1941448d750918183f6dc4) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:33.711083 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac05165-55e6-4339-84f2-7a193ec45ab6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:33.712522 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac05165-55e6-4339-84f2-7a193ec45ab6 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:33.718809 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 623/1246] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:24:33 2022] GET /identity/v3/role_assignments?user.id=75f0586a8c1941448d750918183f6dc4&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=0e34a6f9b89a4e29951fb189abacb34a => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:36.225577 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a206b7c-976b-4f5c-a10e-e0395bab8648 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:36.226219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a206b7c-976b-4f5c-a10e-e0395bab8648 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:36.226219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a206b7c-976b-4f5c-a10e-e0395bab8648 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:36.227503 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 624/1247] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:24:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:24:36.233476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5beb15e-d961-41a6-baa1-733407364a65 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:36.233977 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5beb15e-d961-41a6-baa1-733407364a65 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:36.234097 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5beb15e-d961-41a6-baa1-733407364a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:36.240972 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:36.253806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-b5beb15e-d961-41a6-baa1-733407364a65 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:36.255023 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b5beb15e-d961-41a6-baa1-733407364a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:36.336344 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 624/1248] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:24:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:24:36.363427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a206b7c-976b-4f5c-a10e-e0395bab8648 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:36.387516 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:36.490913 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a206b7c-976b-4f5c-a10e-e0395bab8648 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:36.534720 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5fba6-f4e8-40a6-af6b-5056d50ff2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:36.535703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5fba6-f4e8-40a6-af6b-5056d50ff2dd None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:36.536077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5fba6-f4e8-40a6-af6b-5056d50ff2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:36.536441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5fba6-f4e8-40a6-af6b-5056d50ff2dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:36.549294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f5fba6-f4e8-40a6-af6b-5056d50ff2dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:24:36.557859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-18f5fba6-f4e8-40a6-af6b-5056d50ff2dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:24:36.557859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:36.557859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:36.557859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:36.557859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:36.557859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:36.557859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:36.557859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:36.557859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:36.557859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:36.557859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:36.557859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:36.557859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:24:36.557859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:24:36.557859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:24:36.557859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:24:36.560017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:24:36.560017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:24:36.560017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:24:36.560017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:24:36.560017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:36.560017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:36.560017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:36.560017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:36.560017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:36.560017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:36.560017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:36.560017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:36.560017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:36.560017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:36.560017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:36.560017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:36.561967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:36.561967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:36.561967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:36.561967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:36.561967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:24:36.561967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:24:36.561967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:24:36.561967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:24:36.561967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:24:36.561967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:24:36.561967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:24:36.561967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:24:36.565015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 625/1249] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:24:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:36.566159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5beb15e-d961-41a6-baa1-733407364a65 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:36.678803 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5beb15e-d961-41a6-baa1-733407364a65 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:36.712229 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:36.719005 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bfe98a-6929-4d90-bd66-a0e9c922d029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:36.719741 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfe98a-6929-4d90-bd66-a0e9c922d029 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:36.720048 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfe98a-6929-4d90-bd66-a0e9c922d029 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:36.720387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfe98a-6929-4d90-bd66-a0e9c922d029 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:36.721472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bfe98a-6929-4d90-bd66-a0e9c922d029 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:36.721472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bfe98a-6929-4d90-bd66-a0e9c922d029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:36.722917 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bfe98a-6929-4d90-bd66-a0e9c922d029 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:36.732798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 625/1250] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:24:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:36.740174 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5fba6-f4e8-40a6-af6b-5056d50ff2dd None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:36.829983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:36.856530 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5fba6-f4e8-40a6-af6b-5056d50ff2dd None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:36.916065 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ab176-fd4f-4eb7-9658-3519968a9d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:36.916934 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ab176-fd4f-4eb7-9658-3519968a9d0d None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:36.917260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ab176-fd4f-4eb7-9658-3519968a9d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:36.917649 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ab176-fd4f-4eb7-9658-3519968a9d0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:36.930275 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ab176-fd4f-4eb7-9658-3519968a9d0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:36.931829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ab176-fd4f-4eb7-9658-3519968a9d0d None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:36.953889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-290ab176-fd4f-4eb7-9658-3519968a9d0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:36.953889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:36.953889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:36.953889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:36.953889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:36.953889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:36.953889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:36.953889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:36.953889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:36.953889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:36.953889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:36.953889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:36.953889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:24:36.953889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:24:36.953889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:24:36.955694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:24:36.955694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:36.955694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:36.955694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:36.955694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:36.955694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:36.955694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:36.955694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:36.955694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:36.955694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:36.955694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:36.955694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:36.955694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:36.955694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:36.955694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:36.955694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:36.955694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:36.957459 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:24:36.957459 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:24:36.957459 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:36.957459 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:24:36.957459 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 626/1251] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:36.962040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bfe98a-6929-4d90-bd66-a0e9c922d029 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:37.077271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bfe98a-6929-4d90-bd66-a0e9c922d029 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:37.120348 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebba2a9-4606-456f-9558-faa462abbd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:37.121396 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebba2a9-4606-456f-9558-faa462abbd56 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:37.121821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebba2a9-4606-456f-9558-faa462abbd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:37.122233 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebba2a9-4606-456f-9558-faa462abbd56 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:37.123123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebba2a9-4606-456f-9558-faa462abbd56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:37.123457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebba2a9-4606-456f-9558-faa462abbd56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:37.124858 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebba2a9-4606-456f-9558-faa462abbd56 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:37.138162 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 626/1252] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:37.144206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ab176-fd4f-4eb7-9658-3519968a9d0d None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:37.261901 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ab176-fd4f-4eb7-9658-3519968a9d0d None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:37.289973 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:37.298152 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9619156-29e2-4174-9740-12590ed964b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:37.298964 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9619156-29e2-4174-9740-12590ed964b6 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:37.299252 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9619156-29e2-4174-9740-12590ed964b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:37.299539 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9619156-29e2-4174-9740-12590ed964b6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:37.307557 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9619156-29e2-4174-9740-12590ed964b6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:37.309082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9619156-29e2-4174-9740-12590ed964b6 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:37.325350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-e9619156-29e2-4174-9740-12590ed964b6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:24:37.325350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:37.325350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:37.325350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:37.325350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:37.325350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:37.325350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:37.325350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:37.325350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:37.325350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:37.325350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:37.325350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:37.325350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:24:37.325350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:24:37.325350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:24:37.325350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:24:37.326386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:37.326386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:37.326386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:24:37.326386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:24:37.326386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:24:37.326386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:24:37.326386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:37.326386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:37.326386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:37.326386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:37.326386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:37.326386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:37.326386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:37.326386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:37.326386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:37.326386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:37.326386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:37.326386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:37.332397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:37.332397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:37.332397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:24:37.332397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:24:37.332397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:24:37.332397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:24:37.332397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:24:37.332397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:24:37.332397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:24:37.332397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:24:37.332397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 627/1253] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:24:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:37.333047 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebba2a9-4606-456f-9558-faa462abbd56 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:37.395108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:37.440866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebba2a9-4606-456f-9558-faa462abbd56 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:37.482990 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e1edf-96f6-4242-97d3-d3b0e8092e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:37.483697 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e1edf-96f6-4242-97d3-d3b0e8092e33 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:37.483988 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e1edf-96f6-4242-97d3-d3b0e8092e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:37.484266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e1edf-96f6-4242-97d3-d3b0e8092e33 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:37.486022 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e1edf-96f6-4242-97d3-d3b0e8092e33 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:37.486352 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e1edf-96f6-4242-97d3-d3b0e8092e33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:37.487958 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e1edf-96f6-4242-97d3-d3b0e8092e33 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:37.509525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 627/1254] 10.222.0.40 () {56 vars in 1143 bytes} [Thu Jun 16 17:24:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:37.515389 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9619156-29e2-4174-9740-12590ed964b6 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:37.627825 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9619156-29e2-4174-9740-12590ed964b6 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:37.664202 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c35e8e-79a1-4575-8068-827704eb24d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:37.664862 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c35e8e-79a1-4575-8068-827704eb24d3 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:37.665342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c35e8e-79a1-4575-8068-827704eb24d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:37.665748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c35e8e-79a1-4575-8068-827704eb24d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:37.728002 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:37.742966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c35e8e-79a1-4575-8068-827704eb24d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:37.744428 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c35e8e-79a1-4575-8068-827704eb24d3 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:37.862953 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 628/1255] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:24:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:24:37.871416 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e1edf-96f6-4242-97d3-d3b0e8092e33 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:37.988148 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e1edf-96f6-4242-97d3-d3b0e8092e33 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:38.005074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:38.030470 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a6e35-7e8b-4b6e-a3fc-7a1c435fedc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:38.031295 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a6e35-7e8b-4b6e-a3fc-7a1c435fedc9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:38.031582 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a6e35-7e8b-4b6e-a3fc-7a1c435fedc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:38.031863 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a6e35-7e8b-4b6e-a3fc-7a1c435fedc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:38.043045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a6e35-7e8b-4b6e-a3fc-7a1c435fedc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:38.044533 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a6e35-7e8b-4b6e-a3fc-7a1c435fedc9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:38.063438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-be3a6e35-7e8b-4b6e-a3fc-7a1c435fedc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:38.063438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:38.063438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:38.063438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:38.063438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:38.063438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:38.063438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:38.063438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:38.063438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:38.063438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:38.063438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:38.063438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:38.063438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:24:38.063438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:24:38.063438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:24:38.064607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:24:38.064607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:38.064607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:38.064607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:38.064607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:38.064607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:38.064607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:38.064607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:38.064607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:38.064607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:38.064607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:38.064607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:38.064607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:38.064607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:38.064607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:38.064607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:38.064607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:38.065800 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:24:38.065800 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:24:38.065800 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:38.065800 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:38.067445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 628/1256] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:38.073305 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c35e8e-79a1-4575-8068-827704eb24d3 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:38.175174 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:38.189413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c35e8e-79a1-4575-8068-827704eb24d3 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:38.227192 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4f6c9-8582-40c8-b246-69c8797bbf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:38.227888 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4f6c9-8582-40c8-b246-69c8797bbf03 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:38.228182 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4f6c9-8582-40c8-b246-69c8797bbf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:38.228507 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4f6c9-8582-40c8-b246-69c8797bbf03 None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:38.229261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4f6c9-8582-40c8-b246-69c8797bbf03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:38.229580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4f6c9-8582-40c8-b246-69c8797bbf03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:38.231030 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4f6c9-8582-40c8-b246-69c8797bbf03 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:38.250012 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 629/1257] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:38.251480 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a6e35-7e8b-4b6e-a3fc-7a1c435fedc9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:38.359055 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a6e35-7e8b-4b6e-a3fc-7a1c435fedc9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:38.398289 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f1913-0c04-4b45-af1a-31a825ed708d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:38.399108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f1913-0c04-4b45-af1a-31a825ed708d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:38.399399 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f1913-0c04-4b45-af1a-31a825ed708d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:38.399683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f1913-0c04-4b45-af1a-31a825ed708d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:38.407062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f1913-0c04-4b45-af1a-31a825ed708d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:38.408544 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f1913-0c04-4b45-af1a-31a825ed708d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:38.423135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-c54f1913-0c04-4b45-af1a-31a825ed708d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:24:38.423135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:38.423135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:38.423135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:38.423135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:38.423135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:38.423135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:38.423135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:38.423135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:38.423135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:38.423135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:38.423135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:38.423135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:24:38.423135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:24:38.423135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:24:38.424336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:24:38.424336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:38.424336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:38.424336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:38.424336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:38.424336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:38.424336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:38.424336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:38.424336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:38.424336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:38.424336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:38.424336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:38.424336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:38.424336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:38.424336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:38.424336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:38.424336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:38.425414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:24:38.425414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:24:38.425414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:24:38.425414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:24:38.425414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:24:38.425414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:24:38.425414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:24:38.425414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:38.425414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 629/1258] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:24:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:38.430770 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4f6c9-8582-40c8-b246-69c8797bbf03 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:38.546292 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4f6c9-8582-40c8-b246-69c8797bbf03 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:38.580187 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5b1c9-1253-4326-ab9d-a66ba7330fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:38.580851 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5b1c9-1253-4326-ab9d-a66ba7330fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:38.581135 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5b1c9-1253-4326-ab9d-a66ba7330fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:38.581445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5b1c9-1253-4326-ab9d-a66ba7330fe3 None admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:38.582242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a5b1c9-1253-4326-ab9d-a66ba7330fe3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:38.582560 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a5b1c9-1253-4326-ab9d-a66ba7330fe3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:38.585263 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a5b1c9-1253-4326-ab9d-a66ba7330fe3 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:38.598141 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 630/1259] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:24:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:38.604352 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f1913-0c04-4b45-af1a-31a825ed708d None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:38.631129 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:38.710584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f1913-0c04-4b45-af1a-31a825ed708d None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:38.746110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df827a1c-b5f3-4367-9c24-1cb8dfded89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:38.746786 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df827a1c-b5f3-4367-9c24-1cb8dfded89d None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:38.747078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df827a1c-b5f3-4367-9c24-1cb8dfded89d None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:38.747364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df827a1c-b5f3-4367-9c24-1cb8dfded89d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:38.748219 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df827a1c-b5f3-4367-9c24-1cb8dfded89d None admin] RBAC: Adding query filter params (role.id=25a16d8ea4084e9eacdebb7a4dcd2337, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=75f0586a8c1941448d750918183f6dc4) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:38.748543 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df827a1c-b5f3-4367-9c24-1cb8dfded89d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:38.750003 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df827a1c-b5f3-4367-9c24-1cb8dfded89d None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:38.756829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 630/1260] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:24:38 2022] GET /identity/v3/role_assignments?user.id=75f0586a8c1941448d750918183f6dc4&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=25a16d8ea4084e9eacdebb7a4dcd2337 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:40.904156 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a0990-b1c6-45c9-a6a0-c0301ca81abf None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:40.904755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a0990-b1c6-45c9-a6a0-c0301ca81abf None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:40.904755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a0990-b1c6-45c9-a6a0-c0301ca81abf None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:40.906351 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 631/1261] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:24:40.913530 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6cf529-c2d2-428e-8c22-5f33c3c607f3 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:40.913954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6cf529-c2d2-428e-8c22-5f33c3c607f3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:40.914131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6cf529-c2d2-428e-8c22-5f33c3c607f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:40.933587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-5b6cf529-c2d2-428e-8c22-5f33c3c607f3 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:40.934922 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5b6cf529-c2d2-428e-8c22-5f33c3c607f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:41.008320 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 631/1262] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:24:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:24:41.030945 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a0990-b1c6-45c9-a6a0-c0301ca81abf None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:41.042389 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:41.129749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a0990-b1c6-45c9-a6a0-c0301ca81abf None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:41.163098 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8176c-4841-44c4-8186-eec74ae25a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:41.163819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8176c-4841-44c4-8186-eec74ae25a50 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:41.164105 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8176c-4841-44c4-8186-eec74ae25a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:41.164385 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8176c-4841-44c4-8186-eec74ae25a50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:41.175373 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8176c-4841-44c4-8186-eec74ae25a50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:24:41.183220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-12e8176c-4841-44c4-8186-eec74ae25a50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:24:41.183220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:41.183220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:41.183220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:41.183220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:41.183220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:41.183220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:41.183220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:41.183220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:41.183220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:41.183220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:41.183220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:41.183220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:24:41.183220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:24:41.183220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:24:41.183220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:24:41.184371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:24:41.184371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:24:41.184371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:24:41.184371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:24:41.184371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:41.184371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:41.184371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:41.184371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:41.184371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:41.184371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:41.184371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:41.184371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:41.184371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:41.184371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:41.184371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:41.184371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:41.185455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:41.185455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:41.185455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:41.185455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:41.185455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:24:41.185455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:24:41.185455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:24:41.185455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:24:41.185455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:24:41.185455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:24:41.185455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:24:41.185455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:24:41.185455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 632/1263] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:24:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:41.190626 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6cf529-c2d2-428e-8c22-5f33c3c607f3 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:41.209035 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:41.291486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6cf529-c2d2-428e-8c22-5f33c3c607f3 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:41.324980 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da39574-67c1-4418-9262-65ed0935cfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:41.325676 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da39574-67c1-4418-9262-65ed0935cfc9 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:41.325960 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da39574-67c1-4418-9262-65ed0935cfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:41.326238 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da39574-67c1-4418-9262-65ed0935cfc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:41.326977 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da39574-67c1-4418-9262-65ed0935cfc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:41.327284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da39574-67c1-4418-9262-65ed0935cfc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:41.328642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da39574-67c1-4418-9262-65ed0935cfc9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:41.337276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 632/1264] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:24:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:41.343039 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8176c-4841-44c4-8186-eec74ae25a50 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:41.408439 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:41.440845 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8176c-4841-44c4-8186-eec74ae25a50 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:41.473844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78965263-8a94-46d2-84fc-228a7add4bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:41.474683 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78965263-8a94-46d2-84fc-228a7add4bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:41.474971 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78965263-8a94-46d2-84fc-228a7add4bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:41.475254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78965263-8a94-46d2-84fc-228a7add4bb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:41.485572 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78965263-8a94-46d2-84fc-228a7add4bb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:41.487020 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78965263-8a94-46d2-84fc-228a7add4bb0 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:41.504125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-78965263-8a94-46d2-84fc-228a7add4bb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:41.504125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:41.504125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:41.504125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:41.504125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:41.504125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:41.504125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:41.504125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:41.504125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:41.504125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:41.504125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:41.504125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:41.504125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:24:41.504125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:24:41.504125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:24:41.505886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:24:41.505886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:41.505886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:41.505886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:41.505886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:41.505886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:41.505886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:41.505886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:41.505886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:41.505886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:41.505886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:41.505886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:41.505886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:41.505886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:41.505886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:41.505886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:41.505886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:41.506894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:24:41.506894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:24:41.506894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:41.506894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:24:41.506894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 633/1265] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:41.511658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da39574-67c1-4418-9262-65ed0935cfc9 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:41.590549 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:41.609275 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da39574-67c1-4418-9262-65ed0935cfc9 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:41.642946 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e80d8-3c37-44e9-a3dd-169544dfc5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:41.643604 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e80d8-3c37-44e9-a3dd-169544dfc5ac None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:41.643886 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e80d8-3c37-44e9-a3dd-169544dfc5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:41.644161 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e80d8-3c37-44e9-a3dd-169544dfc5ac None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:41.644881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e80d8-3c37-44e9-a3dd-169544dfc5ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:41.645213 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e80d8-3c37-44e9-a3dd-169544dfc5ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:41.647154 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e80d8-3c37-44e9-a3dd-169544dfc5ac None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:41.658603 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 633/1266] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:41.664802 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78965263-8a94-46d2-84fc-228a7add4bb0 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:41.763966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78965263-8a94-46d2-84fc-228a7add4bb0 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:41.783184 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:41.797539 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff594f-2fc4-4ad4-874e-a1b1e7946157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:41.798337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff594f-2fc4-4ad4-874e-a1b1e7946157 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:41.798621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff594f-2fc4-4ad4-874e-a1b1e7946157 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:41.798898 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff594f-2fc4-4ad4-874e-a1b1e7946157 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:41.806259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff594f-2fc4-4ad4-874e-a1b1e7946157 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:41.807681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff594f-2fc4-4ad4-874e-a1b1e7946157 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:41.821580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-e7ff594f-2fc4-4ad4-874e-a1b1e7946157 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:24:41.821580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:41.821580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:41.821580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:41.821580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:41.821580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:41.821580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:41.821580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:41.821580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:41.821580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:41.821580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:41.821580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:41.821580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:24:41.821580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:24:41.821580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:24:41.821580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:24:41.822731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:41.822731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:41.822731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:24:41.822731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:24:41.822731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:24:41.822731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:24:41.822731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:41.822731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:41.822731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:41.822731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:41.822731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:41.822731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:41.822731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:41.822731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:41.822731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:41.822731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:41.822731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:41.822731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:41.824118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:41.824118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:41.824118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:24:41.824118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:24:41.824118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:24:41.824118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:24:41.824118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:24:41.824118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:24:41.824118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:24:41.824118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:24:41.824118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 634/1267] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:24:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:41.828536 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e80d8-3c37-44e9-a3dd-169544dfc5ac None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:41.939490 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e80d8-3c37-44e9-a3dd-169544dfc5ac None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:41.972968 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9660d18-aa96-4c1c-85b0-dae47df1bf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:41.973699 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9660d18-aa96-4c1c-85b0-dae47df1bf65 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:41.973983 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9660d18-aa96-4c1c-85b0-dae47df1bf65 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:41.974368 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9660d18-aa96-4c1c-85b0-dae47df1bf65 None admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:41.975092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9660d18-aa96-4c1c-85b0-dae47df1bf65 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:41.975898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9660d18-aa96-4c1c-85b0-dae47df1bf65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:41.977347 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9660d18-aa96-4c1c-85b0-dae47df1bf65 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:41.985706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:41.995165 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 634/1268] 10.222.0.40 () {56 vars in 1143 bytes} [Thu Jun 16 17:24:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:42.001076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff594f-2fc4-4ad4-874e-a1b1e7946157 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:42.101596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff594f-2fc4-4ad4-874e-a1b1e7946157 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:42.134852 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19af0d-1f29-4587-9ccc-b0f7429c1b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:42.135635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19af0d-1f29-4587-9ccc-b0f7429c1b5b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:42.135919 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19af0d-1f29-4587-9ccc-b0f7429c1b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:42.136195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19af0d-1f29-4587-9ccc-b0f7429c1b5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:42.146826 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19af0d-1f29-4587-9ccc-b0f7429c1b5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:42.148250 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19af0d-1f29-4587-9ccc-b0f7429c1b5b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:42.165526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-6b19af0d-1f29-4587-9ccc-b0f7429c1b5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:42.165526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:42.165526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:42.165526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:42.165526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:42.165526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:42.165526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:42.165526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:42.165526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:42.165526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:42.165526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:42.165526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:42.165526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:24:42.165526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:24:42.165526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:24:42.166635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:24:42.166635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:42.166635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:42.166635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:42.166635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:42.166635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:42.166635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:42.166635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:42.166635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:42.166635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:42.166635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:42.166635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:42.166635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:42.166635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:42.166635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:42.166635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:42.166635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:42.167703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:24:42.167703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:24:42.167703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:42.167703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:24:42.167703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 635/1269] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:42.172360 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9660d18-aa96-4c1c-85b0-dae47df1bf65 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:42.270756 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9660d18-aa96-4c1c-85b0-dae47df1bf65 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:42.303934 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee5438-26df-4e06-9a5f-8959285a8a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:42.304590 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee5438-26df-4e06-9a5f-8959285a8a3e None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:42.304875 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee5438-26df-4e06-9a5f-8959285a8a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:42.305177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee5438-26df-4e06-9a5f-8959285a8a3e None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:42.305906 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee5438-26df-4e06-9a5f-8959285a8a3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:42.306215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee5438-26df-4e06-9a5f-8959285a8a3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:42.307572 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee5438-26df-4e06-9a5f-8959285a8a3e None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:42.318587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 635/1270] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:42.324185 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19af0d-1f29-4587-9ccc-b0f7429c1b5b None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:42.355384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:42.421793 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19af0d-1f29-4587-9ccc-b0f7429c1b5b None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:42.454311 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4502a-0596-49b3-a9f0-1ac4bd23a57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:42.455099 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4502a-0596-49b3-a9f0-1ac4bd23a57f None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:42.455389 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4502a-0596-49b3-a9f0-1ac4bd23a57f None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:42.455672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4502a-0596-49b3-a9f0-1ac4bd23a57f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:42.462717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4502a-0596-49b3-a9f0-1ac4bd23a57f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:42.464153 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4502a-0596-49b3-a9f0-1ac4bd23a57f None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:42.478617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-5fa4502a-0596-49b3-a9f0-1ac4bd23a57f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:24:42.478617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:42.478617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:42.478617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:42.478617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:42.478617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:42.478617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:42.478617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:42.478617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:42.478617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:42.478617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:42.478617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:42.478617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:24:42.478617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:24:42.478617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:24:42.479682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:24:42.479682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:42.479682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:42.479682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:42.479682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:42.479682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:42.479682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:42.479682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:42.479682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:42.479682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:42.479682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:42.479682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:42.479682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:42.479682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:42.479682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:42.479682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:42.479682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:42.480808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:24:42.480808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:24:42.480808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:24:42.480808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:24:42.480808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:24:42.480808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:24:42.480808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:24:42.480808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:24:42.480808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 636/1271] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:24:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:42.485571 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee5438-26df-4e06-9a5f-8959285a8a3e None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:42.545976 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:42.588780 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee5438-26df-4e06-9a5f-8959285a8a3e None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:42.622918 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59a384-da0b-45bf-bf6b-45267c765e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:42.623518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59a384-da0b-45bf-bf6b-45267c765e06 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:42.623810 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59a384-da0b-45bf-bf6b-45267c765e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:42.624092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59a384-da0b-45bf-bf6b-45267c765e06 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:42.625283 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59a384-da0b-45bf-bf6b-45267c765e06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:42.625606 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59a384-da0b-45bf-bf6b-45267c765e06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:42.627036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59a384-da0b-45bf-bf6b-45267c765e06 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:42.639324 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 636/1272] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:24:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:42.645946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4502a-0596-49b3-a9f0-1ac4bd23a57f None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:42.735881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:42.745133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4502a-0596-49b3-a9f0-1ac4bd23a57f None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:42.779513 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ea5e9-2a4a-49aa-ba09-c8a244913572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:42.780400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ea5e9-2a4a-49aa-ba09-c8a244913572 None admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:42.780692 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ea5e9-2a4a-49aa-ba09-c8a244913572 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:42.780975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ea5e9-2a4a-49aa-ba09-c8a244913572 None admin] PATH_INFO: `/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/75f0586a8c1941448d750918183f6dc4/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:42.782952 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ea5e9-2a4a-49aa-ba09-c8a244913572 None admin] RBAC: Authorizing `identity:create_grant(project_id=35a535a305e740368b7a4d77d0ad22a0, user_id=75f0586a8c1941448d750918183f6dc4, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:42.784380 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ea5e9-2a4a-49aa-ba09-c8a244913572 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:42.798245 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 637/1273] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 17:24:42 2022] PUT /identity/v3/projects/35a535a305e740368b7a4d77d0ad22a0/users/75f0586a8c1941448d750918183f6dc4/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:44.906297 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bda72e-3169-4b8b-bb7f-b097cb799b8b None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:44.906866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bda72e-3169-4b8b-bb7f-b097cb799b8b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:44.906866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bda72e-3169-4b8b-bb7f-b097cb799b8b None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:44.908317 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 637/1274] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:24:44.914465 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c2295-97f7-4d68-af9c-7dd3f2ea2e28 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:44.914950 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c2295-97f7-4d68-af9c-7dd3f2ea2e28 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:44.915209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c2295-97f7-4d68-af9c-7dd3f2ea2e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:44.934250 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-a20c2295-97f7-4d68-af9c-7dd3f2ea2e28 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:44.935643 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-a20c2295-97f7-4d68-af9c-7dd3f2ea2e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:45.005365 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 638/1275] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:24:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:24:45.030536 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bda72e-3169-4b8b-bb7f-b097cb799b8b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:45.131689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bda72e-3169-4b8b-bb7f-b097cb799b8b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:45.148688 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:45.165789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae3c95-ea62-4b2c-96a7-d0795daca1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:45.166524 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae3c95-ea62-4b2c-96a7-d0795daca1bb None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:45.166816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae3c95-ea62-4b2c-96a7-d0795daca1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:45.167099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae3c95-ea62-4b2c-96a7-d0795daca1bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:45.177705 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae3c95-ea62-4b2c-96a7-d0795daca1bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:24:45.185364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-beae3c95-ea62-4b2c-96a7-d0795daca1bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:24:45.185364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:45.185364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:45.185364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:45.185364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:45.185364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:45.185364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:45.185364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:45.185364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:45.185364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:45.185364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:45.185364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:45.185364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:24:45.185364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:24:45.185364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:24:45.185364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:24:45.187039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:24:45.187039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:24:45.187039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:24:45.187039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:24:45.187039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:45.187039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:45.187039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:45.187039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:45.187039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:45.187039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:45.187039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:45.187039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:45.187039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:45.187039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:45.187039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:45.187039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:45.188751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:45.188751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:45.188751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:45.188751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:45.188751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:24:45.188751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:24:45.188751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:24:45.188751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:24:45.188751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:24:45.188751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:24:45.188751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:24:45.188751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:45.191481 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 638/1276] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 17:24:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:45.192666 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c2295-97f7-4d68-af9c-7dd3f2ea2e28 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:45.277891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:45.299994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c2295-97f7-4d68-af9c-7dd3f2ea2e28 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:45.335827 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133274d-7e1b-4f3f-b0c5-00e8721ce0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:45.336502 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133274d-7e1b-4f3f-b0c5-00e8721ce0ea None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:45.336796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133274d-7e1b-4f3f-b0c5-00e8721ce0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:45.337079 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133274d-7e1b-4f3f-b0c5-00e8721ce0ea None admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:45.337869 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1133274d-7e1b-4f3f-b0c5-00e8721ce0ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:45.338183 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1133274d-7e1b-4f3f-b0c5-00e8721ce0ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:45.339551 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1133274d-7e1b-4f3f-b0c5-00e8721ce0ea None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:45.347514 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 639/1277] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 17:24:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:45.354058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae3c95-ea62-4b2c-96a7-d0795daca1bb None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:45.455490 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae3c95-ea62-4b2c-96a7-d0795daca1bb None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:45.490553 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ad0c9-3371-40df-b88e-97ec343660d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:45.491337 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ad0c9-3371-40df-b88e-97ec343660d5 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:45.491629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ad0c9-3371-40df-b88e-97ec343660d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:45.491916 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ad0c9-3371-40df-b88e-97ec343660d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:45.502536 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ad0c9-3371-40df-b88e-97ec343660d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:45.503969 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ad0c9-3371-40df-b88e-97ec343660d5 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:45.521966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-9e9ad0c9-3371-40df-b88e-97ec343660d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:45.521966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:45.521966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:45.521966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:45.521966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:45.521966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:45.521966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:45.521966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:45.521966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:45.521966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:45.521966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:45.521966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:45.521966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:24:45.521966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:24:45.521966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:24:45.522939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:24:45.522939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:45.522939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:45.522939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:45.522939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:45.522939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:45.522939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:45.522939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:45.522939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:45.522939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:45.522939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:45.522939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:45.522939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:45.522939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:45.522939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:45.522939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:45.522939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:45.523932 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:24:45.523932 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:24:45.523932 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:45.523932 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:45.523932 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 639/1278] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:24:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:45.528844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133274d-7e1b-4f3f-b0c5-00e8721ce0ea None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:45.631178 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133274d-7e1b-4f3f-b0c5-00e8721ce0ea None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:45.665066 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e51dc0-7e22-4c70-8e2a-dc98f3a3f2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:45.665751 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e51dc0-7e22-4c70-8e2a-dc98f3a3f2da None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:45.666038 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e51dc0-7e22-4c70-8e2a-dc98f3a3f2da None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:45.666317 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e51dc0-7e22-4c70-8e2a-dc98f3a3f2da None admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:45.667046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e51dc0-7e22-4c70-8e2a-dc98f3a3f2da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:45.667356 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e51dc0-7e22-4c70-8e2a-dc98f3a3f2da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:45.668718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e51dc0-7e22-4c70-8e2a-dc98f3a3f2da None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:45.680313 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 640/1279] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:45.686084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ad0c9-3371-40df-b88e-97ec343660d5 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:45.718646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:45.795743 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ad0c9-3371-40df-b88e-97ec343660d5 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:45.834020 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f377965-8e61-4e31-91f6-69e0a45f435b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:45.834848 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f377965-8e61-4e31-91f6-69e0a45f435b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:45.835138 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f377965-8e61-4e31-91f6-69e0a45f435b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:45.835423 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f377965-8e61-4e31-91f6-69e0a45f435b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:45.843384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f377965-8e61-4e31-91f6-69e0a45f435b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:45.844828 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f377965-8e61-4e31-91f6-69e0a45f435b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:45.860725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-5f377965-8e61-4e31-91f6-69e0a45f435b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:24:45.860725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:45.860725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:45.860725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:45.860725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:45.860725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:45.860725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:45.860725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:45.860725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:45.860725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:45.860725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:45.860725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:45.860725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:24:45.860725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:24:45.860725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:24:45.860725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:24:45.861979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:45.861979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:45.861979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:24:45.861979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:24:45.861979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:24:45.861979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:24:45.861979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:45.861979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:45.861979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:45.861979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:45.861979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:45.861979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:45.861979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:45.861979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:45.861979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:45.861979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:45.861979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:45.861979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:45.863049 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:45.863049 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:45.863049 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:24:45.863049 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:24:45.863049 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:24:45.863049 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:24:45.863049 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:24:45.863049 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:24:45.863049 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:24:45.863049 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:45.863049 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 640/1280] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:24:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:24:45.868984 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e51dc0-7e22-4c70-8e2a-dc98f3a3f2da None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:45.873074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:45.974688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e51dc0-7e22-4c70-8e2a-dc98f3a3f2da None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:46.010771 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84202aa-b873-46ab-b6e2-216da41f36f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:46.011357 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84202aa-b873-46ab-b6e2-216da41f36f2 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:46.011650 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84202aa-b873-46ab-b6e2-216da41f36f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:46.011937 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84202aa-b873-46ab-b6e2-216da41f36f2 None admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:46.012795 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84202aa-b873-46ab-b6e2-216da41f36f2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:46.013113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84202aa-b873-46ab-b6e2-216da41f36f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:46.014586 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84202aa-b873-46ab-b6e2-216da41f36f2 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:46.032443 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 641/1281] 10.222.0.40 () {56 vars in 1143 bytes} [Thu Jun 16 17:24:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:24:46.039593 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f377965-8e61-4e31-91f6-69e0a45f435b None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:46.133890 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:46.144416 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f377965-8e61-4e31-91f6-69e0a45f435b None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:46.180693 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6d3ab-870d-46ca-9d3b-9c18e45cc384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:46.181376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6d3ab-870d-46ca-9d3b-9c18e45cc384 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:46.181670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6d3ab-870d-46ca-9d3b-9c18e45cc384 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:46.181954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6d3ab-870d-46ca-9d3b-9c18e45cc384 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:46.250420 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6d3ab-870d-46ca-9d3b-9c18e45cc384 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:46.252044 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6d3ab-870d-46ca-9d3b-9c18e45cc384 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:46.339793 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:46.350899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 641/1282] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 17:24:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:24:46.356785 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84202aa-b873-46ab-b6e2-216da41f36f2 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:46.417834 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:46.455900 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84202aa-b873-46ab-b6e2-216da41f36f2 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:46.490704 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc53e11-ee00-4a35-b982-38d4a3fa5d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:46.491440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc53e11-ee00-4a35-b982-38d4a3fa5d23 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:46.491731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc53e11-ee00-4a35-b982-38d4a3fa5d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:46.492019 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc53e11-ee00-4a35-b982-38d4a3fa5d23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:46.502247 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc53e11-ee00-4a35-b982-38d4a3fa5d23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:46.503724 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc53e11-ee00-4a35-b982-38d4a3fa5d23 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:46.522097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-7fc53e11-ee00-4a35-b982-38d4a3fa5d23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:46.522097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:46.522097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:46.522097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:46.522097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:46.522097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:46.522097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:46.522097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:46.522097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:46.522097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:46.522097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:46.522097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:46.522097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:24:46.522097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:24:46.522097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:24:46.523229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:24:46.523229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:46.523229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:46.523229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:46.523229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:46.523229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:46.523229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:46.523229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:46.523229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:46.523229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:46.523229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:46.523229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:46.523229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:46.523229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:46.523229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:46.523229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:46.523229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:46.524537 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:24:46.524537 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:24:46.524537 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:24:46.524537 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:24:46.524537 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 642/1283] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:46.528954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6d3ab-870d-46ca-9d3b-9c18e45cc384 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:46.626484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6d3ab-870d-46ca-9d3b-9c18e45cc384 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:46.659777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73e5f6-4b03-498b-89bf-0e62061b10c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:46.660428 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73e5f6-4b03-498b-89bf-0e62061b10c4 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:46.660712 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73e5f6-4b03-498b-89bf-0e62061b10c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:46.660984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73e5f6-4b03-498b-89bf-0e62061b10c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:46.661856 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73e5f6-4b03-498b-89bf-0e62061b10c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:46.662162 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73e5f6-4b03-498b-89bf-0e62061b10c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:46.663517 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73e5f6-4b03-498b-89bf-0e62061b10c4 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:46.674345 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 642/1284] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 17:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:46.680511 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc53e11-ee00-4a35-b982-38d4a3fa5d23 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:46.782082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc53e11-ee00-4a35-b982-38d4a3fa5d23 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:46.795831 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:46.815845 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75e361-edd3-4651-b13b-dd0fc0dfb804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:46.816641 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75e361-edd3-4651-b13b-dd0fc0dfb804 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:46.816932 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75e361-edd3-4651-b13b-dd0fc0dfb804 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:46.817251 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75e361-edd3-4651-b13b-dd0fc0dfb804 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:46.824644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75e361-edd3-4651-b13b-dd0fc0dfb804 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:46.826119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75e361-edd3-4651-b13b-dd0fc0dfb804 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:46.840441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application [None req-ec75e361-edd3-4651-b13b-dd0fc0dfb804 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:24:46.840441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:46.840441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:46.840441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:46.840441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:46.840441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:46.840441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:46.840441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:46.840441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:46.840441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:46.840441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:46.840441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:46.840441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:24:46.840441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:24:46.840441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:24:46.841567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:24:46.841567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:46.841567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:46.841567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:46.841567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:46.841567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:46.841567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:46.841567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:46.841567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:46.841567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:46.841567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:46.841567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:46.841567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:46.841567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:46.841567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:46.841567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:46.841567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:46.842728 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:24:46.842728 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:24:46.842728 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:24:46.842728 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:24:46.842728 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:24:46.842728 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:24:46.842728 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:24:46.842728 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: ERROR keystone.server.flask.application Jun 16 17:24:46.842728 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 643/1285] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 17:24:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:46.847437 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73e5f6-4b03-498b-89bf-0e62061b10c4 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:46.902583 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:46.957242 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73e5f6-4b03-498b-89bf-0e62061b10c4 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:46.991106 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f0c05-2a1c-4019-88f2-04caf588d561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:46.991776 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f0c05-2a1c-4019-88f2-04caf588d561 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:46.992066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f0c05-2a1c-4019-88f2-04caf588d561 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:46.992346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f0c05-2a1c-4019-88f2-04caf588d561 None admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:46.993139 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f0c05-2a1c-4019-88f2-04caf588d561 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:46.993484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f0c05-2a1c-4019-88f2-04caf588d561 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:46.994919 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f0c05-2a1c-4019-88f2-04caf588d561 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:47.007692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 643/1286] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 17:24:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:47.013942 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75e361-edd3-4651-b13b-dd0fc0dfb804 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:47.111459 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75e361-edd3-4651-b13b-dd0fc0dfb804 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:47.144882 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c594d8c-fd91-44fc-a719-2aa95a475125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:47.145563 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c594d8c-fd91-44fc-a719-2aa95a475125 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:47.145935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c594d8c-fd91-44fc-a719-2aa95a475125 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:47.146274 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c594d8c-fd91-44fc-a719-2aa95a475125 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:47.147055 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c594d8c-fd91-44fc-a719-2aa95a475125 None admin] RBAC: Adding query filter params (role.id=25a16d8ea4084e9eacdebb7a4dcd2337, scope.project.id=35a535a305e740368b7a4d77d0ad22a0, user.id=75f0586a8c1941448d750918183f6dc4) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:47.147375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c594d8c-fd91-44fc-a719-2aa95a475125 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:47.148800 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c594d8c-fd91-44fc-a719-2aa95a475125 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:47.154548 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 644/1287] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 17:24:47 2022] GET /identity/v3/role_assignments?user.id=75f0586a8c1941448d750918183f6dc4&scope.project.id=35a535a305e740368b7a4d77d0ad22a0&role.id=25a16d8ea4084e9eacdebb7a4dcd2337 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:49.341961 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1aa40-bc69-4c00-a03e-84f8958bd589 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:49.342523 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1aa40-bc69-4c00-a03e-84f8958bd589 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:49.342523 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1aa40-bc69-4c00-a03e-84f8958bd589 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:49.343920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 644/1288] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:24:49.349436 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071822e-796b-4479-8a6e-21a8a6549d79 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:49.349898 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071822e-796b-4479-8a6e-21a8a6549d79 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:49.350189 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071822e-796b-4479-8a6e-21a8a6549d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:49.367361 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-3071822e-796b-4479-8a6e-21a8a6549d79 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:49.368546 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-3071822e-796b-4479-8a6e-21a8a6549d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:49.397894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:49.434544 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 645/1289] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:24:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:24:49.457071 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f1aa40-bc69-4c00-a03e-84f8958bd589 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:49.556989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f1aa40-bc69-4c00-a03e-84f8958bd589 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:49.567369 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:49.590326 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4fccf8-211a-4a86-92cd-8f3f4aecd818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:49.591130 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4fccf8-211a-4a86-92cd-8f3f4aecd818 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:49.591418 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4fccf8-211a-4a86-92cd-8f3f4aecd818 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:49.591697 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4fccf8-211a-4a86-92cd-8f3f4aecd818 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:49.592353 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4fccf8-211a-4a86-92cd-8f3f4aecd818 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:49.593756 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4fccf8-211a-4a86-92cd-8f3f4aecd818 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:49.605967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-7c4fccf8-211a-4a86-92cd-8f3f4aecd818 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:24:49.605967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:49.605967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:24:49.605967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:24:49.605967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:24:49.605967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:24:49.605967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:24:49.605967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:24:49.605967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:24:49.605967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:49.605967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:24:49.605967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:49.605967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:49.605967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:49.605967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:49.605967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:49.605967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:49.607346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:49.607346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:49.607346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:49.607346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:49.607346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:49.607346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:49.607346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:24:49.607346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:24:49.607346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:24:49.607346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:24:49.607346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:49.607346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:49.607346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:49.607346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:49.607346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:49.607346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:49.608276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:49.608276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:49.608276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:49.608276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:49.608276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:49.608276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:49.608276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:49.608276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:49.608276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:49.608276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:49.608276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:24:49.608276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:24:49.608276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:24:49.608276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:49.608276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 645/1290] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jun 16 17:24:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:49.613044 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071822e-796b-4479-8a6e-21a8a6549d79 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:49.709908 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071822e-796b-4479-8a6e-21a8a6549d79 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:49.744894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b64cff-d17c-460c-a955-06b1d7ac097b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:49.745586 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b64cff-d17c-460c-a955-06b1d7ac097b None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:49.745878 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b64cff-d17c-460c-a955-06b1d7ac097b None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:49.746160 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b64cff-d17c-460c-a955-06b1d7ac097b None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:49.746872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b64cff-d17c-460c-a955-06b1d7ac097b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:49.747185 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b64cff-d17c-460c-a955-06b1d7ac097b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:49.748553 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b64cff-d17c-460c-a955-06b1d7ac097b None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:49.753641 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 646/1291] 10.222.0.40 () {56 vars in 1116 bytes} [Thu Jun 16 17:24:49 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:49.759552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4fccf8-211a-4a86-92cd-8f3f4aecd818 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:49.857884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4fccf8-211a-4a86-92cd-8f3f4aecd818 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:49.893820 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab1338-344c-4cc8-851e-f15820948b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:49.894508 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab1338-344c-4cc8-851e-f15820948b7b None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:49.894771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab1338-344c-4cc8-851e-f15820948b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:49.895058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab1338-344c-4cc8-851e-f15820948b7b None admin] PATH_INFO: `/v3/services` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:49.895899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab1338-344c-4cc8-851e-f15820948b7b None admin] RBAC: Adding query filter params (type=placement) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:49.896406 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab1338-344c-4cc8-851e-f15820948b7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:49.897683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab1338-344c-4cc8-851e-f15820948b7b None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:49.902462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 646/1292] 10.222.0.40 () {56 vars in 1116 bytes} [Thu Jun 16 17:24:49 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:52.026469 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b472174-9a34-4525-bcde-250e60380dc5 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:52.027052 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b472174-9a34-4525-bcde-250e60380dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:52.027052 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b472174-9a34-4525-bcde-250e60380dc5 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:52.028387 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 647/1293] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:24:52.033750 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba3d62-8758-4e32-a3e2-2749174c01ec None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:52.034180 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba3d62-8758-4e32-a3e2-2749174c01ec None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:52.034318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba3d62-8758-4e32-a3e2-2749174c01ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:52.051227 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0eba3d62-8758-4e32-a3e2-2749174c01ec None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:52.053082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0eba3d62-8758-4e32-a3e2-2749174c01ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:52.082074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:52.120295 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 647/1294] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:24:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:24:52.143367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b472174-9a34-4525-bcde-250e60380dc5 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:52.174873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:52.245370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b472174-9a34-4525-bcde-250e60380dc5 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:52.278957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a96d92-c1ad-463b-a269-6262732784e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:52.279621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a96d92-c1ad-463b-a269-6262732784e0 None admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:52.279915 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a96d92-c1ad-463b-a269-6262732784e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:52.280202 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a96d92-c1ad-463b-a269-6262732784e0 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:52.280948 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a96d92-c1ad-463b-a269-6262732784e0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:52.282434 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a96d92-c1ad-463b-a269-6262732784e0 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:52.295590 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 648/1295] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 17:24:52 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:24:54.397753 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ede076-eb34-465b-81de-06243223e89d None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:54.398296 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ede076-eb34-465b-81de-06243223e89d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:54.398296 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ede076-eb34-465b-81de-06243223e89d None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:54.399689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 648/1296] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:24:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:24:54.404866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d995434-3e1f-4b7e-9e64-15f6c6ee0325 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:54.405302 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d995434-3e1f-4b7e-9e64-15f6c6ee0325 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:54.405468 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d995434-3e1f-4b7e-9e64-15f6c6ee0325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:54.422578 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-0d995434-3e1f-4b7e-9e64-15f6c6ee0325 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:54.423908 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0d995434-3e1f-4b7e-9e64-15f6c6ee0325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:54.500831 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 649/1297] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:24:54 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:24:54.524477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ede076-eb34-465b-81de-06243223e89d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:54.626040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ede076-eb34-465b-81de-06243223e89d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:54.659687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7816b6-d4a0-4ea2-b0b1-36b5bfca0b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:54.660485 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7816b6-d4a0-4ea2-b0b1-36b5bfca0b8c None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:54.660779 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7816b6-d4a0-4ea2-b0b1-36b5bfca0b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:54.661697 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7816b6-d4a0-4ea2-b0b1-36b5bfca0b8c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:54.662359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7816b6-d4a0-4ea2-b0b1-36b5bfca0b8c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:54.663725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7816b6-d4a0-4ea2-b0b1-36b5bfca0b8c None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:54.675365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-bb7816b6-d4a0-4ea2-b0b1-36b5bfca0b8c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:24:54.675365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:54.675365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:24:54.675365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:24:54.675365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:24:54.675365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:24:54.675365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:24:54.675365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:24:54.675365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:24:54.675365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:54.675365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:24:54.675365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:54.675365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:54.675365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:54.675365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:54.675365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:54.675365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:54.676679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:54.676679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:54.676679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:54.676679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:54.676679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:54.676679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:54.676679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:24:54.676679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:24:54.676679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:24:54.676679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:24:54.676679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:54.676679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:54.676679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:54.676679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:54.676679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:54.676679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:54.677816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:54.677816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:54.677816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:54.677816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:54.677816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:54.677816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:54.677816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:54.677816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:54.677816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:54.677816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:54.677816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:24:54.677816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:24:54.677816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:24:54.677816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:54.677816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 649/1298] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jun 16 17:24:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:54.682891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d995434-3e1f-4b7e-9e64-15f6c6ee0325 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:54.701391 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:54.788356 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d995434-3e1f-4b7e-9e64-15f6c6ee0325 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:54.823929 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800d320-5ce2-4cf4-9e1a-827433c2ded1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:54.824964 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800d320-5ce2-4cf4-9e1a-827433c2ded1 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:54.825292 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800d320-5ce2-4cf4-9e1a-827433c2ded1 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:54.825583 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800d320-5ce2-4cf4-9e1a-827433c2ded1 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:54.826311 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800d320-5ce2-4cf4-9e1a-827433c2ded1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:54.826632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800d320-5ce2-4cf4-9e1a-827433c2ded1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:54.828007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800d320-5ce2-4cf4-9e1a-827433c2ded1 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:54.834458 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 650/1299] 10.222.0.40 () {56 vars in 1116 bytes} [Thu Jun 16 17:24:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:54.840694 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7816b6-d4a0-4ea2-b0b1-36b5bfca0b8c None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:54.872499 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:54.940189 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7816b6-d4a0-4ea2-b0b1-36b5bfca0b8c None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:54.973293 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f489ea52-37b5-4e84-87f4-76d933090091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:54.973976 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489ea52-37b5-4e84-87f4-76d933090091 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:54.974265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489ea52-37b5-4e84-87f4-76d933090091 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:54.974542 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489ea52-37b5-4e84-87f4-76d933090091 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:54.975340 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f489ea52-37b5-4e84-87f4-76d933090091 None admin] RBAC: Adding query filter params (interface=public, service_id=a7b446af663f48f1b296f24b997178ad, region_id=RegionOne) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:54.975658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f489ea52-37b5-4e84-87f4-76d933090091 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:54.977019 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f489ea52-37b5-4e84-87f4-76d933090091 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:54.981971 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 650/1300] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jun 16 17:24:54 2022] GET /identity/v3/endpoints?service_id=a7b446af663f48f1b296f24b997178ad&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:54.987910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800d320-5ce2-4cf4-9e1a-827433c2ded1 None admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:55.069640 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:55.099664 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800d320-5ce2-4cf4-9e1a-827433c2ded1 None admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:55.137341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9be61-662d-48f3-a990-909e4ea2b1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:55.137999 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9be61-662d-48f3-a990-909e4ea2b1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:55.138286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9be61-662d-48f3-a990-909e4ea2b1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:55.138565 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9be61-662d-48f3-a990-909e4ea2b1e2 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:55.139247 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9be61-662d-48f3-a990-909e4ea2b1e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:55.140603 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9be61-662d-48f3-a990-909e4ea2b1e2 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:55.148118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 651/1301] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jun 16 17:24:54 2022] GET /identity/v3/services => generated 2191 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:57.256234 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b23534-92d5-4a72-8843-a646a8034f4c None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:57.256718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b23534-92d5-4a72-8843-a646a8034f4c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:57.256826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b23534-92d5-4a72-8843-a646a8034f4c None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:57.258194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 651/1302] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:24:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:24:57.263262 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6531d-e89b-484f-a313-0b74d09dcb42 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:57.263670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6531d-e89b-484f-a313-0b74d09dcb42 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:57.263833 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6531d-e89b-484f-a313-0b74d09dcb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:57.280695 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-18a6531d-e89b-484f-a313-0b74d09dcb42 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:57.281990 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-18a6531d-e89b-484f-a313-0b74d09dcb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:57.347383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 652/1303] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:24:57 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:24:57.371080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b23534-92d5-4a72-8843-a646a8034f4c None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:57.468322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:57.475735 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b23534-92d5-4a72-8843-a646a8034f4c None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:57.511066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21861935-6a78-461b-a6b2-0133c642a620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:57.511799 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21861935-6a78-461b-a6b2-0133c642a620 None admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:57.512089 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21861935-6a78-461b-a6b2-0133c642a620 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:57.512374 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21861935-6a78-461b-a6b2-0133c642a620 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:57.513030 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21861935-6a78-461b-a6b2-0133c642a620 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:57.514431 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21861935-6a78-461b-a6b2-0133c642a620 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:57.526655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application [None req-21861935-6a78-461b-a6b2-0133c642a620 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:24:57.526655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:57.526655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:24:57.526655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:24:57.526655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:24:57.526655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:24:57.526655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:24:57.526655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:24:57.526655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:24:57.526655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:57.526655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:24:57.526655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:57.526655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:24:57.526655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:24:57.526655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:24:57.526655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:24:57.526655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:24:57.527859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:24:57.527859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:24:57.527859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:24:57.527859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:24:57.527859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:24:57.527859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:24:57.527859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:24:57.527859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:24:57.527859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:24:57.527859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:24:57.527859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:24:57.527859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:24:57.527859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:24:57.527859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:24:57.527859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:24:57.527859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:24:57.529078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:24:57.529078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application with Lock( Jun 16 17:24:57.529078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:24:57.529078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:24:57.529078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:24:57.529078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:24:57.529078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:24:57.529078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:24:57.529078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:24:57.529078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:24:57.529078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:24:57.529078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:24:57.529078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:24:57.529078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: ERROR keystone.server.flask.application Jun 16 17:24:57.529078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 652/1304] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jun 16 17:24:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:24:57.533859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6531d-e89b-484f-a313-0b74d09dcb42 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:57.607300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:57.630927 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6531d-e89b-484f-a313-0b74d09dcb42 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:57.663399 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d3023-147e-4e42-a28a-a066867ae1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:57.664056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d3023-147e-4e42-a28a-a066867ae1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:57.664343 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d3023-147e-4e42-a28a-a066867ae1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:57.664626 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d3023-147e-4e42-a28a-a066867ae1d4 None admin] PATH_INFO: `/v3/services` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:57.665359 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d3023-147e-4e42-a28a-a066867ae1d4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:57.665674 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d3023-147e-4e42-a28a-a066867ae1d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:57.667030 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d3023-147e-4e42-a28a-a066867ae1d4 None admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:57.672109 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 653/1305] 10.222.0.40 () {56 vars in 1116 bytes} [Thu Jun 16 17:24:57 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:57.678653 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21861935-6a78-461b-a6b2-0133c642a620 None admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:57.777559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21861935-6a78-461b-a6b2-0133c642a620 None admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:57.810941 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c0445-85ad-4135-9e1f-3ea6b2424cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:57.811619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c0445-85ad-4135-9e1f-3ea6b2424cd9 None admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:57.811913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c0445-85ad-4135-9e1f-3ea6b2424cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:57.812198 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c0445-85ad-4135-9e1f-3ea6b2424cd9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:57.812953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c0445-85ad-4135-9e1f-3ea6b2424cd9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:57.814361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c0445-85ad-4135-9e1f-3ea6b2424cd9 None admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:57.831866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 653/1306] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jun 16 17:24:57 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:25:07.669341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=30 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:25:07.827502 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:01.940445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141f6d4-7d4a-4db9-90e0-24bd6e19a91a None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:01.940918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141f6d4-7d4a-4db9-90e0-24bd6e19a91a None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:01.941220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141f6d4-7d4a-4db9-90e0-24bd6e19a91a None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:01.952598 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27db433-0914-4eec-82a2-827f02b88c41 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:01.953137 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27db433-0914-4eec-82a2-827f02b88c41 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:01.953137 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27db433-0914-4eec-82a2-827f02b88c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:01.957240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 654/1307] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:26:01 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:26:02.044011 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a27db433-0914-4eec-82a2-827f02b88c41 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:02.045861 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a27db433-0914-4eec-82a2-827f02b88c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:02.297054 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 654/1308] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:02.359223 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c97b17-b5fd-41e2-9fcb-35bd873be4b8 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:02.359546 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c97b17-b5fd-41e2-9fcb-35bd873be4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:02.359904 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c97b17-b5fd-41e2-9fcb-35bd873be4b8 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:02.362004 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 655/1309] 10.222.0.40 () {54 vars in 964 bytes} [Thu Jun 16 17:26:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:26:02.375571 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba0990-d3d2-4d4a-b2aa-144274db0200 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:02.376096 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba0990-d3d2-4d4a-b2aa-144274db0200 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:02.377006 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba0990-d3d2-4d4a-b2aa-144274db0200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:02.506625 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-8fba0990-d3d2-4d4a-b2aa-144274db0200 None None] MFA Rules not processed for user `a303489493ce4cd2b62e876fbc8a7404`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:02.507876 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8fba0990-d3d2-4d4a-b2aa-144274db0200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:02.531601 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:02.602858 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 655/1310] 10.222.0.40 () {58 vars in 1061 bytes} [Thu Jun 16 17:26:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:02.603817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c97b17-b5fd-41e2-9fcb-35bd873be4b8 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:26:02.605928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-c1c97b17-b5fd-41e2-9fcb-35bd873be4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:02.740135 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c97b17-b5fd-41e2-9fcb-35bd873be4b8 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:26:02.826989 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423bc06-f224-4709-bf68-757bf1a9bc56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:26:02.827959 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423bc06-f224-4709-bf68-757bf1a9bc56 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:02.828405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423bc06-f224-4709-bf68-757bf1a9bc56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:02.828853 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423bc06-f224-4709-bf68-757bf1a9bc56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:02.831433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-2423bc06-f224-4709-bf68-757bf1a9bc56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:02.899797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:02.959773 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423bc06-f224-4709-bf68-757bf1a9bc56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:26:02.961256 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423bc06-f224-4709-bf68-757bf1a9bc56 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:26:02.997644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 656/1311] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:26:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:26:05.488587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e912fcf-3109-4a14-b8ff-b7c0235c447c None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:05.489106 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e912fcf-3109-4a14-b8ff-b7c0235c447c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:05.489106 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e912fcf-3109-4a14-b8ff-b7c0235c447c None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:05.490479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 656/1312] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:26:05.495975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc2245-eb40-49a7-a925-10140d226b51 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:05.496622 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc2245-eb40-49a7-a925-10140d226b51 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:05.496622 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc2245-eb40-49a7-a925-10140d226b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:05.578538 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-cbfc2245-eb40-49a7-a925-10140d226b51 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:05.580479 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-cbfc2245-eb40-49a7-a925-10140d226b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:05.683845 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 657/1313] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:05.743014 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526e453-0613-4173-8954-759907f241d5 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:05.743211 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526e453-0613-4173-8954-759907f241d5 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:05.743498 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526e453-0613-4173-8954-759907f241d5 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:05.745206 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 657/1314] 10.222.0.40 () {54 vars in 964 bytes} [Thu Jun 16 17:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:26:05.751998 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec403d51-fcea-43f1-bd90-ecbaca89bfec None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:05.752290 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec403d51-fcea-43f1-bd90-ecbaca89bfec None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:05.752577 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec403d51-fcea-43f1-bd90-ecbaca89bfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:05.763633 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:05.839187 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-ec403d51-fcea-43f1-bd90-ecbaca89bfec None None] MFA Rules not processed for user `a303489493ce4cd2b62e876fbc8a7404`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:05.840490 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-ec403d51-fcea-43f1-bd90-ecbaca89bfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:05.929468 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 658/1315] 10.222.0.40 () {58 vars in 1061 bytes} [Thu Jun 16 17:26:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:05.943040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526e453-0613-4173-8954-759907f241d5 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:26:05.944662 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1526e453-0613-4173-8954-759907f241d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:06.062734 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526e453-0613-4173-8954-759907f241d5 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:26:06.143634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:06.146427 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0587444-f10f-4103-877f-444c6de95c9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:26:06.147357 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0587444-f10f-4103-877f-444c6de95c9c service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:06.147971 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0587444-f10f-4103-877f-444c6de95c9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:06.148444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0587444-f10f-4103-877f-444c6de95c9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:06.150193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b0587444-f10f-4103-877f-444c6de95c9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:06.269209 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0587444-f10f-4103-877f-444c6de95c9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:26:06.270674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0587444-f10f-4103-877f-444c6de95c9c service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:26:06.290946 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 658/1316] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:26:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:26:08.955440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83744081-9674-4bcb-8aef-db1683683a28 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:08.956052 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83744081-9674-4bcb-8aef-db1683683a28 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:08.956052 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83744081-9674-4bcb-8aef-db1683683a28 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:08.957391 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 659/1317] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:26:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:26:08.962517 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7058f-c8b1-444d-bcb6-065a48ce58f8 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:08.962935 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7058f-c8b1-444d-bcb6-065a48ce58f8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:08.963075 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7058f-c8b1-444d-bcb6-065a48ce58f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:09.027803 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d1b7058f-c8b1-444d-bcb6-065a48ce58f8 None None] MFA Rules not processed for user `ed0016d7b075461eb857ad7527a36809`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:09.029006 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d1b7058f-c8b1-444d-bcb6-065a48ce58f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:09.086059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:09.131676 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 659/1318] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:26:08 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:09.175227 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83744081-9674-4bcb-8aef-db1683683a28 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:26:09.176783 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-83744081-9674-4bcb-8aef-db1683683a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:09.289330 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83744081-9674-4bcb-8aef-db1683683a28 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:26:09.292173 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:09.367370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92e9a7-2733-4213-99f4-b55c3c12cc36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:26:09.368023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92e9a7-2733-4213-99f4-b55c3c12cc36 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:09.368313 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92e9a7-2733-4213-99f4-b55c3c12cc36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:09.368595 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92e9a7-2733-4213-99f4-b55c3c12cc36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:09.370564 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-7f92e9a7-2733-4213-99f4-b55c3c12cc36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:09.485378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f92e9a7-2733-4213-99f4-b55c3c12cc36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:26:09.486716 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f92e9a7-2733-4213-99f4-b55c3c12cc36 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:26:09.508070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 660/1319] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:26:09 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:26:12.313577 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18182e-cf69-43e6-95e4-9d5c1955803e None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:12.314147 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18182e-cf69-43e6-95e4-9d5c1955803e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:12.314609 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18182e-cf69-43e6-95e4-9d5c1955803e None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:12.316472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 660/1320] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:26:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:26:12.321608 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33e1cb-6d6f-40b9-8ac9-c6da93baa2ac None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:12.322068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33e1cb-6d6f-40b9-8ac9-c6da93baa2ac None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:12.322166 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33e1cb-6d6f-40b9-8ac9-c6da93baa2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:12.372792 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:12.412046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-2d33e1cb-6d6f-40b9-8ac9-c6da93baa2ac None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:12.414037 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-2d33e1cb-6d6f-40b9-8ac9-c6da93baa2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:12.523942 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 661/1321] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:26:12 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:12.562601 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18182e-cf69-43e6-95e4-9d5c1955803e None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:26:12.564922 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ab18182e-cf69-43e6-95e4-9d5c1955803e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:12.666761 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18182e-cf69-43e6-95e4-9d5c1955803e None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:26:12.753065 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113eaaf-5f06-4e92-b64f-4dc72ac77e7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:26:12.753753 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113eaaf-5f06-4e92-b64f-4dc72ac77e7b service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:12.754041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113eaaf-5f06-4e92-b64f-4dc72ac77e7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:12.754320 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113eaaf-5f06-4e92-b64f-4dc72ac77e7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:12.755981 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4113eaaf-5f06-4e92-b64f-4dc72ac77e7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:12.763475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:12.864045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113eaaf-5f06-4e92-b64f-4dc72ac77e7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:26:12.865550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113eaaf-5f06-4e92-b64f-4dc72ac77e7b service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:26:12.883979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 661/1322] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:26:12 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:26:15.714971 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462833b-f6db-44b4-b7dd-2c3568ddc59d None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:15.715510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462833b-f6db-44b4-b7dd-2c3568ddc59d None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:15.715510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462833b-f6db-44b4-b7dd-2c3568ddc59d None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:15.716897 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 662/1323] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:26:15.722582 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209641a-0659-4d7b-9115-c8adfeb9720b None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:15.723006 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209641a-0659-4d7b-9115-c8adfeb9720b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:15.723258 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209641a-0659-4d7b-9115-c8adfeb9720b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:15.795854 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-f209641a-0659-4d7b-9115-c8adfeb9720b None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:15.797295 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f209641a-0659-4d7b-9115-c8adfeb9720b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:15.867010 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:15.917158 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 662/1324] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:15.953916 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462833b-f6db-44b4-b7dd-2c3568ddc59d None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:26:15.956009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-e462833b-f6db-44b4-b7dd-2c3568ddc59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:16.063865 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462833b-f6db-44b4-b7dd-2c3568ddc59d None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:26:16.076134 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:16.141833 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d52e09-8e4b-4a1c-b0f0-743487ac6fa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:26:16.142499 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d52e09-8e4b-4a1c-b0f0-743487ac6fa9 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:16.142793 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d52e09-8e4b-4a1c-b0f0-743487ac6fa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:16.143078 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d52e09-8e4b-4a1c-b0f0-743487ac6fa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:16.144761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-e7d52e09-8e4b-4a1c-b0f0-743487ac6fa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:16.260707 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d52e09-8e4b-4a1c-b0f0-743487ac6fa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:26:16.262229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d52e09-8e4b-4a1c-b0f0-743487ac6fa9 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:26:16.282142 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 663/1325] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:26:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:26:18.486898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16f6da-c22a-4c6b-9f21-f1c009f48d7e None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:18.487444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16f6da-c22a-4c6b-9f21-f1c009f48d7e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:18.487444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16f6da-c22a-4c6b-9f21-f1c009f48d7e None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:18.488918 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 663/1326] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:26:18.494402 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1bae2c-b01a-411a-9b85-4022fcd0e6b5 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:18.494826 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1bae2c-b01a-411a-9b85-4022fcd0e6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:18.494979 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1bae2c-b01a-411a-9b85-4022fcd0e6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:18.544275 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:18.564010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-7d1bae2c-b01a-411a-9b85-4022fcd0e6b5 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:18.565239 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-7d1bae2c-b01a-411a-9b85-4022fcd0e6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:18.663671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 664/1327] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:18.699006 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16f6da-c22a-4c6b-9f21-f1c009f48d7e None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:26:18.700559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2c16f6da-c22a-4c6b-9f21-f1c009f48d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:18.815666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16f6da-c22a-4c6b-9f21-f1c009f48d7e None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:26:18.898747 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1031b-bde8-4f2e-a84f-bd4699fc51dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:26:18.899508 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1031b-bde8-4f2e-a84f-bd4699fc51dc service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:18.899795 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1031b-bde8-4f2e-a84f-bd4699fc51dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:18.900083 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1031b-bde8-4f2e-a84f-bd4699fc51dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:18.901762 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-05c1031b-bde8-4f2e-a84f-bd4699fc51dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:18.918842 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:19.025006 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1031b-bde8-4f2e-a84f-bd4699fc51dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:26:19.026491 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1031b-bde8-4f2e-a84f-bd4699fc51dc service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:26:19.046201 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 664/1328] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:26:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:26:21.243444 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b6d1b-f669-46f2-95ef-70d696e96215 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:21.243873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b6d1b-f669-46f2-95ef-70d696e96215 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:21.244330 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b6d1b-f669-46f2-95ef-70d696e96215 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:21.245836 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 665/1329] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:26:21.251617 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1923b5c-2bb9-4214-8443-8f663e8ae5be None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:21.252058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1923b5c-2bb9-4214-8443-8f663e8ae5be None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:21.252185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1923b5c-2bb9-4214-8443-8f663e8ae5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:21.319295 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d1923b5c-2bb9-4214-8443-8f663e8ae5be None None] MFA Rules not processed for user `ed0016d7b075461eb857ad7527a36809`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:21.320501 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d1923b5c-2bb9-4214-8443-8f663e8ae5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:21.386731 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:21.422460 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 665/1330] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:26:21 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:21.465845 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b6d1b-f669-46f2-95ef-70d696e96215 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:26:21.467411 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-363b6d1b-f669-46f2-95ef-70d696e96215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:21.575724 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b6d1b-f669-46f2-95ef-70d696e96215 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:26:21.600652 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:21.666983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f04e0a0-0e18-470a-b9d2-c9ea77d6adc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:26:21.667650 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04e0a0-0e18-470a-b9d2-c9ea77d6adc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:21.667940 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04e0a0-0e18-470a-b9d2-c9ea77d6adc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:21.668220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04e0a0-0e18-470a-b9d2-c9ea77d6adc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:21.669958 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-9f04e0a0-0e18-470a-b9d2-c9ea77d6adc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:21.793867 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f04e0a0-0e18-470a-b9d2-c9ea77d6adc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:26:21.795329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f04e0a0-0e18-470a-b9d2-c9ea77d6adc8 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:26:21.815040 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 666/1331] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:26:21 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:26:24.274350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87d314-8503-4963-832a-4e39f04d67d5 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:24.274885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87d314-8503-4963-832a-4e39f04d67d5 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:24.275002 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87d314-8503-4963-832a-4e39f04d67d5 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:24.276326 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 666/1332] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:26:24.282380 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21466f47-4539-4147-972e-45d0e9e60c73 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:24.282853 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21466f47-4539-4147-972e-45d0e9e60c73 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:24.282963 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21466f47-4539-4147-972e-45d0e9e60c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:24.346771 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:24.349738 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-21466f47-4539-4147-972e-45d0e9e60c73 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:24.351549 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-21466f47-4539-4147-972e-45d0e9e60c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:24.447312 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 667/1333] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:24.490051 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87d314-8503-4963-832a-4e39f04d67d5 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:26:24.491592 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ff87d314-8503-4963-832a-4e39f04d67d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:24.594458 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87d314-8503-4963-832a-4e39f04d67d5 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:26:24.665842 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfc29f-1235-4b32-af18-1ddbe297022d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:26:24.666499 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfc29f-1235-4b32-af18-1ddbe297022d service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:24.666780 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfc29f-1235-4b32-af18-1ddbe297022d service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:24.667057 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfc29f-1235-4b32-af18-1ddbe297022d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:24.668692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4cdfc29f-1235-4b32-af18-1ddbe297022d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:24.696235 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:24.777967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfc29f-1235-4b32-af18-1ddbe297022d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:26:24.779354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfc29f-1235-4b32-af18-1ddbe297022d service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:26:24.798633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 667/1334] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:26:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:26:26.975397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d5bc2-e155-4c8c-b50c-7a11188ff02e None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:26.975951 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d5bc2-e155-4c8c-b50c-7a11188ff02e None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:26.975951 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d5bc2-e155-4c8c-b50c-7a11188ff02e None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:26.977390 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 668/1335] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:26:26.982570 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b113a05-0ffe-4454-804d-e94fbc1f41b3 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:26.983054 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b113a05-0ffe-4454-804d-e94fbc1f41b3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:26.983159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b113a05-0ffe-4454-804d-e94fbc1f41b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:27.047966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-6b113a05-0ffe-4454-804d-e94fbc1f41b3 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:27.049207 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6b113a05-0ffe-4454-804d-e94fbc1f41b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:27.123919 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:27.148507 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 668/1336] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:27.189686 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d5bc2-e155-4c8c-b50c-7a11188ff02e None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:26:27.191270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-697d5bc2-e155-4c8c-b50c-7a11188ff02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:27.299404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d5bc2-e155-4c8c-b50c-7a11188ff02e None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:26:27.328672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:27.374734 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355fd15-8ee2-4e63-a8c8-f9757a8ee752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:26:27.375485 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355fd15-8ee2-4e63-a8c8-f9757a8ee752 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:27.375771 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355fd15-8ee2-4e63-a8c8-f9757a8ee752 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:27.376051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355fd15-8ee2-4e63-a8c8-f9757a8ee752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:27.377771 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b355fd15-8ee2-4e63-a8c8-f9757a8ee752 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:27.495762 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b355fd15-8ee2-4e63-a8c8-f9757a8ee752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:26:27.497330 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b355fd15-8ee2-4e63-a8c8-f9757a8ee752 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:26:27.519445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 669/1337] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:26:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:26:30.006765 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf3082-d75e-4036-bb6f-91052f9982bc None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:30.007316 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf3082-d75e-4036-bb6f-91052f9982bc None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:30.007316 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf3082-d75e-4036-bb6f-91052f9982bc None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:30.008725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 669/1338] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:26:30.014052 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55be0b-d486-4e01-ab7c-e1181fac1acd None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:30.014443 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55be0b-d486-4e01-ab7c-e1181fac1acd None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:30.014853 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55be0b-d486-4e01-ab7c-e1181fac1acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:30.080793 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-3c55be0b-d486-4e01-ab7c-e1181fac1acd None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:30.082035 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-3c55be0b-d486-4e01-ab7c-e1181fac1acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:30.090305 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:30.183255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 670/1339] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:26:30 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:30.220498 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf3082-d75e-4036-bb6f-91052f9982bc None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:26:30.222078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-17cf3082-d75e-4036-bb6f-91052f9982bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:30.326779 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf3082-d75e-4036-bb6f-91052f9982bc None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:26:30.401113 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb45865-71a9-4198-a425-19f44321a628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:26:30.401855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb45865-71a9-4198-a425-19f44321a628 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:30.402145 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb45865-71a9-4198-a425-19f44321a628 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:30.402426 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb45865-71a9-4198-a425-19f44321a628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:30.404083 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-beb45865-71a9-4198-a425-19f44321a628 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:30.440179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:30.513021 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb45865-71a9-4198-a425-19f44321a628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:26:30.514533 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb45865-71a9-4198-a425-19f44321a628 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:26:30.532356 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 670/1340] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:26:30 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:26:34.708183 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b42619-8878-4978-a427-f666c388f754 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:34.708691 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b42619-8878-4978-a427-f666c388f754 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:34.708691 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b42619-8878-4978-a427-f666c388f754 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:34.710062 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 671/1341] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:26:34.715273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d341028-0e31-4926-888a-012eda1f48c4 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:34.715738 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d341028-0e31-4926-888a-012eda1f48c4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:34.715963 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d341028-0e31-4926-888a-012eda1f48c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:34.788348 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-3d341028-0e31-4926-888a-012eda1f48c4 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:34.789588 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3d341028-0e31-4926-888a-012eda1f48c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:34.885553 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:34.899852 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 671/1342] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:34.939987 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b42619-8878-4978-a427-f666c388f754 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:26:34.942169 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-44b42619-8878-4978-a427-f666c388f754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:35.041168 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b42619-8878-4978-a427-f666c388f754 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:26:35.046144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62f966-e49a-48b3-8cc7-0575eb3c3675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:26:35.047277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62f966-e49a-48b3-8cc7-0575eb3c3675 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:35.047571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62f966-e49a-48b3-8cc7-0575eb3c3675 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:35.047857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62f966-e49a-48b3-8cc7-0575eb3c3675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:35.057184 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62f966-e49a-48b3-8cc7-0575eb3c3675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:26:35.058634 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62f966-e49a-48b3-8cc7-0575eb3c3675 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:26:35.068695 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 672/1343] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:26:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:26:37.913434 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a64a32-b6ca-43ff-aaf0-23f59f407cc3 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:37.914115 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a64a32-b6ca-43ff-aaf0-23f59f407cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:37.914115 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a64a32-b6ca-43ff-aaf0-23f59f407cc3 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:37.915381 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 672/1344] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:26:37.921169 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f09aa8-3ba9-4645-8344-67111a1f5860 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:37.921736 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f09aa8-3ba9-4645-8344-67111a1f5860 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:37.921736 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f09aa8-3ba9-4645-8344-67111a1f5860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:37.950771 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-f6f09aa8-3ba9-4645-8344-67111a1f5860 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:37.952977 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-f6f09aa8-3ba9-4645-8344-67111a1f5860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:37.986507 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:38.039299 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 673/1345] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:26:37 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:38.077538 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a64a32-b6ca-43ff-aaf0-23f59f407cc3 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:26:38.088395 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a64a32-b6ca-43ff-aaf0-23f59f407cc3 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:26:38.091891 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b6555-d6b8-4536-9451-cbfb0bf2706a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:26:38.093373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b6555-d6b8-4536-9451-cbfb0bf2706a service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:38.093687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b6555-d6b8-4536-9451-cbfb0bf2706a service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:38.093970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b6555-d6b8-4536-9451-cbfb0bf2706a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:38.103206 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b6555-d6b8-4536-9451-cbfb0bf2706a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:26:38.104632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b6555-d6b8-4536-9451-cbfb0bf2706a service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:26:38.122421 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 673/1346] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:26:38 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:26:42.219908 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c4cc9-8339-49a0-a3ba-990ec3e8d9bf None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:42.220474 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c4cc9-8339-49a0-a3ba-990ec3e8d9bf None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:42.220474 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c4cc9-8339-49a0-a3ba-990ec3e8d9bf None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:42.221914 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 674/1347] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:26:42.227732 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b75a73-9997-4083-9b43-05c2227df6c8 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:42.228196 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b75a73-9997-4083-9b43-05c2227df6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:42.228431 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b75a73-9997-4083-9b43-05c2227df6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:42.247790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-12b75a73-9997-4083-9b43-05c2227df6c8 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:42.249069 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-12b75a73-9997-4083-9b43-05c2227df6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:42.257081 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 674/1348] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:42.296327 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c4cc9-8339-49a0-a3ba-990ec3e8d9bf None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:26:42.298072 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-ed4c4cc9-8339-49a0-a3ba-990ec3e8d9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:42.311572 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c4cc9-8339-49a0-a3ba-990ec3e8d9bf None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:26:42.315206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb4676-ef17-4159-a33b-03ad0fd7cfb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:26:42.315867 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb4676-ef17-4159-a33b-03ad0fd7cfb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:42.316160 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb4676-ef17-4159-a33b-03ad0fd7cfb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:42.316446 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb4676-ef17-4159-a33b-03ad0fd7cfb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:42.325803 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb4676-ef17-4159-a33b-03ad0fd7cfb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:26:42.327402 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb4676-ef17-4159-a33b-03ad0fd7cfb8 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:26:42.337178 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 675/1349] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:26:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:26:47.830427 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955d9c5-1e62-403e-82b9-4299cf27b1ca None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:47.830988 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955d9c5-1e62-403e-82b9-4299cf27b1ca None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:47.830988 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955d9c5-1e62-403e-82b9-4299cf27b1ca None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:47.832605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 675/1350] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 17:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:26:47.842745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeec81b-11b9-4f89-beef-b291de20d9b8 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:47.844602 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeec81b-11b9-4f89-beef-b291de20d9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:47.844898 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeec81b-11b9-4f89-beef-b291de20d9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:47.977245 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-fbeec81b-11b9-4f89-beef-b291de20d9b8 None None] MFA Rules not processed for user `75f0586a8c1941448d750918183f6dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:47.978735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-fbeec81b-11b9-4f89-beef-b291de20d9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:48.066893 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 676/1351] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 17:26:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:52.242263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=13 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:52.368037 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c544e2-b0a8-4909-8548-8771955d2e93 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:52.368384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c544e2-b0a8-4909-8548-8771955d2e93 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:52.368606 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c544e2-b0a8-4909-8548-8771955d2e93 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:52.370219 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 676/1352] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 17:26:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:26:52.376495 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c0ec7-dbe0-4705-935f-f68f05681dbf None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:52.377048 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c0ec7-dbe0-4705-935f-f68f05681dbf None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:52.377143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c0ec7-dbe0-4705-935f-f68f05681dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:52.401315 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-119c0ec7-dbe0-4705-935f-f68f05681dbf None None] MFA Rules not processed for user `75f0586a8c1941448d750918183f6dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:52.402541 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-119c0ec7-dbe0-4705-935f-f68f05681dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:52.413169 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 677/1353] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 17:26:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:53.986839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589795b-929a-4466-a5c4-e3a16e33dd85 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:53.987425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589795b-929a-4466-a5c4-e3a16e33dd85 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:53.987425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589795b-929a-4466-a5c4-e3a16e33dd85 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:53.988855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 677/1354] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 17:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:26:53.994334 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e15885-4e7a-4637-a512-327e626dfc38 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:53.994745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e15885-4e7a-4637-a512-327e626dfc38 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:53.995029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e15885-4e7a-4637-a512-327e626dfc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:54.014155 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-15e15885-4e7a-4637-a512-327e626dfc38 None None] MFA Rules not processed for user `75f0586a8c1941448d750918183f6dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:54.015437 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-15e15885-4e7a-4637-a512-327e626dfc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:54.024686 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 678/1355] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 17:26:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:57.815985 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d48e69-df69-4f78-bad9-a17b966369b6 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:57.817009 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d48e69-df69-4f78-bad9-a17b966369b6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:57.817611 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d48e69-df69-4f78-bad9-a17b966369b6 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:57.819529 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 678/1356] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:26:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:26:57.825792 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a40f4-b4d7-4918-b41d-84c483149c1b None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:57.826312 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a40f4-b4d7-4918-b41d-84c483149c1b None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:57.826599 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a40f4-b4d7-4918-b41d-84c483149c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:57.847368 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-302a40f4-b4d7-4918-b41d-84c483149c1b None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:57.848568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-302a40f4-b4d7-4918-b41d-84c483149c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:57.857758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 679/1357] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:57.913936 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4533880-7a59-4cd1-b052-b946871f16e9 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:57.914655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4533880-7a59-4cd1-b052-b946871f16e9 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:57.915157 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4533880-7a59-4cd1-b052-b946871f16e9 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:57.917050 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 679/1358] 10.222.0.40 () {54 vars in 961 bytes} [Thu Jun 16 17:26:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:26:57.925550 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6695bb5-6d52-4a0e-b3ca-ffa7841a2408 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:57.926186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6695bb5-6d52-4a0e-b3ca-ffa7841a2408 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:57.926786 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6695bb5-6d52-4a0e-b3ca-ffa7841a2408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:57.974048 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:57.983989 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-f6695bb5-6d52-4a0e-b3ca-ffa7841a2408 None None] MFA Rules not processed for user `689a31e26c6b454a91c3fd79678b4d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:57.985228 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-f6695bb5-6d52-4a0e-b3ca-ffa7841a2408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:58.081066 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 680/1359] 10.222.0.40 () {58 vars in 1058 bytes} [Thu Jun 16 17:26:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:26:58.094763 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4533880-7a59-4cd1-b052-b946871f16e9 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:26:58.111022 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4533880-7a59-4cd1-b052-b946871f16e9 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:26:58.116880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40b059-d47a-479d-aa12-a89ca2500aba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:26:58.117572 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40b059-d47a-479d-aa12-a89ca2500aba service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:58.117863 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40b059-d47a-479d-aa12-a89ca2500aba service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:58.118147 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40b059-d47a-479d-aa12-a89ca2500aba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:58.127393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40b059-d47a-479d-aa12-a89ca2500aba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:26:58.128986 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40b059-d47a-479d-aa12-a89ca2500aba service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:26:58.140918 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 680/1360] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:00.585884 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd2133-09aa-4c10-8a65-a2cfbc5aa84e None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:00.586425 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd2133-09aa-4c10-8a65-a2cfbc5aa84e None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:00.586425 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd2133-09aa-4c10-8a65-a2cfbc5aa84e None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:00.587829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 681/1361] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:00.593443 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c6c30-2005-4967-b5d4-1ff5ff55981f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:00.593895 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c6c30-2005-4967-b5d4-1ff5ff55981f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:00.594006 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c6c30-2005-4967-b5d4-1ff5ff55981f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:00.612921 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e84c6c30-2005-4967-b5d4-1ff5ff55981f None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:00.614174 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e84c6c30-2005-4967-b5d4-1ff5ff55981f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:00.625297 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 681/1362] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:00.667201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd2133-09aa-4c10-8a65-a2cfbc5aa84e None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:00.682898 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd2133-09aa-4c10-8a65-a2cfbc5aa84e None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:00.689666 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3db2b-cfea-43e7-a3d2-0ac4d0432f0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:00.690550 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3db2b-cfea-43e7-a3d2-0ac4d0432f0b service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:00.690916 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3db2b-cfea-43e7-a3d2-0ac4d0432f0b service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:00.691275 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3db2b-cfea-43e7-a3d2-0ac4d0432f0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:00.701924 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3db2b-cfea-43e7-a3d2-0ac4d0432f0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:00.703782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3db2b-cfea-43e7-a3d2-0ac4d0432f0b service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:00.716016 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 682/1363] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:03.237773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112ed68-1fd2-46a7-942c-d77daf69030b None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:03.237773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112ed68-1fd2-46a7-942c-d77daf69030b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:03.237773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112ed68-1fd2-46a7-942c-d77daf69030b None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:03.238914 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 682/1364] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:03.244471 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fed6f-5b71-4dbb-abec-b49ee0db41f2 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:03.244906 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fed6f-5b71-4dbb-abec-b49ee0db41f2 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:03.245104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fed6f-5b71-4dbb-abec-b49ee0db41f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:03.264445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-2e4fed6f-5b71-4dbb-abec-b49ee0db41f2 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:03.265692 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-2e4fed6f-5b71-4dbb-abec-b49ee0db41f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:03.273699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 683/1365] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:03.320485 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc49a8c-74a3-4fcb-a4a9-6d93593fe6bb None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:03.320874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc49a8c-74a3-4fcb-a4a9-6d93593fe6bb None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:03.321268 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc49a8c-74a3-4fcb-a4a9-6d93593fe6bb None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:03.322725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 683/1366] 10.222.0.40 () {54 vars in 961 bytes} [Thu Jun 16 17:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:03.328955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88858352-ef07-41d3-bdf3-2c1cd09f0dcc None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:03.329292 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88858352-ef07-41d3-bdf3-2c1cd09f0dcc None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:03.329697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88858352-ef07-41d3-bdf3-2c1cd09f0dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:03.348512 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-88858352-ef07-41d3-bdf3-2c1cd09f0dcc None None] MFA Rules not processed for user `689a31e26c6b454a91c3fd79678b4d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:03.349763 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-88858352-ef07-41d3-bdf3-2c1cd09f0dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:03.365428 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 684/1367] 10.222.0.40 () {58 vars in 1058 bytes} [Thu Jun 16 17:27:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:03.374431 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc49a8c-74a3-4fcb-a4a9-6d93593fe6bb None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:03.388944 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc49a8c-74a3-4fcb-a4a9-6d93593fe6bb None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:03.394469 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab7b68-156c-494c-842d-549708eaa3d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:03.395196 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab7b68-156c-494c-842d-549708eaa3d5 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:03.395549 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab7b68-156c-494c-842d-549708eaa3d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:03.395915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab7b68-156c-494c-842d-549708eaa3d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:03.404318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab7b68-156c-494c-842d-549708eaa3d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:03.406000 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab7b68-156c-494c-842d-549708eaa3d5 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:03.415681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 684/1368] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:05.663568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85189f4-ba07-4b93-9607-0dc628d5efa0 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:05.664108 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85189f4-ba07-4b93-9607-0dc628d5efa0 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:05.664108 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85189f4-ba07-4b93-9607-0dc628d5efa0 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:05.665858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 685/1369] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:05.671429 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c6b7d-3941-4ea1-8545-2f6d2569c8ff None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:05.671865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c6b7d-3941-4ea1-8545-2f6d2569c8ff None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:05.672040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c6b7d-3941-4ea1-8545-2f6d2569c8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:05.691628 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-2a2c6b7d-3941-4ea1-8545-2f6d2569c8ff None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:05.692851 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2a2c6b7d-3941-4ea1-8545-2f6d2569c8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:05.701955 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 685/1370] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:05.747052 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85189f4-ba07-4b93-9607-0dc628d5efa0 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:05.762369 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85189f4-ba07-4b93-9607-0dc628d5efa0 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:05.768061 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164cc21-47ab-4f0b-8ec1-56df057c10eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:05.768872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164cc21-47ab-4f0b-8ec1-56df057c10eb service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:05.769180 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164cc21-47ab-4f0b-8ec1-56df057c10eb service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:05.769482 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164cc21-47ab-4f0b-8ec1-56df057c10eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:05.778964 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164cc21-47ab-4f0b-8ec1-56df057c10eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:05.780569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164cc21-47ab-4f0b-8ec1-56df057c10eb service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:05.790498 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 686/1371] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:06.325777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c955514-6cf7-475c-b843-e3360458925a None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:06.326069 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c955514-6cf7-475c-b843-e3360458925a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:06.326518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c955514-6cf7-475c-b843-e3360458925a None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:06.327914 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 686/1372] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:27:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:06.333250 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cc2a8-fa54-4a89-bfa1-7e80caf94e0b None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:06.333549 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cc2a8-fa54-4a89-bfa1-7e80caf94e0b None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:06.333831 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cc2a8-fa54-4a89-bfa1-7e80caf94e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:06.352133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-c91cc2a8-fa54-4a89-bfa1-7e80caf94e0b None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:06.353364 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-c91cc2a8-fa54-4a89-bfa1-7e80caf94e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:06.363521 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 687/1373] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:06.674236 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59b920-553c-4d45-bcd2-58174f7ff25d None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:06.674798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59b920-553c-4d45-bcd2-58174f7ff25d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:06.675006 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59b920-553c-4d45-bcd2-58174f7ff25d None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:06.676642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 687/1374] 10.222.0.40 () {54 vars in 963 bytes} [Thu Jun 16 17:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:06.682461 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca197f-4744-496f-adc2-5d2c13272e2c None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:06.682461 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca197f-4744-496f-adc2-5d2c13272e2c None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:06.682461 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca197f-4744-496f-adc2-5d2c13272e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:06.735585 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-65ca197f-4744-496f-adc2-5d2c13272e2c None None] MFA Rules not processed for user `1fadecafbe2549acaa4cde5412638d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:06.736802 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-65ca197f-4744-496f-adc2-5d2c13272e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:06.801120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 688/1375] 10.222.0.40 () {58 vars in 1060 bytes} [Thu Jun 16 17:27:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:06.809409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59b920-553c-4d45-bcd2-58174f7ff25d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:06.820457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59b920-553c-4d45-bcd2-58174f7ff25d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:06.824150 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c06aea-fb8f-4681-9263-bd681e4bfb12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:06.824811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c06aea-fb8f-4681-9263-bd681e4bfb12 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:06.825098 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c06aea-fb8f-4681-9263-bd681e4bfb12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:06.825409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c06aea-fb8f-4681-9263-bd681e4bfb12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:06.834786 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c06aea-fb8f-4681-9263-bd681e4bfb12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:06.836223 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c06aea-fb8f-4681-9263-bd681e4bfb12 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:06.846647 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 688/1376] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:08.002852 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1cd1c-136b-4552-8ec3-f175adcacac9 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:08.003366 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1cd1c-136b-4552-8ec3-f175adcacac9 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:08.003366 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1cd1c-136b-4552-8ec3-f175adcacac9 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:08.004741 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 689/1377] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:27:08.010007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a66bf1-3afc-48fa-b4a1-ff11de79c51f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:08.010414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a66bf1-3afc-48fa-b4a1-ff11de79c51f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:08.010713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a66bf1-3afc-48fa-b4a1-ff11de79c51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:08.032546 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-08a66bf1-3afc-48fa-b4a1-ff11de79c51f None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:08.033792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-08a66bf1-3afc-48fa-b4a1-ff11de79c51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:08.041762 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 689/1378] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:08.086287 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1cd1c-136b-4552-8ec3-f175adcacac9 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:08.100872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1cd1c-136b-4552-8ec3-f175adcacac9 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:08.106932 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac04753-107d-4586-89a5-d05578557c8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:08.107573 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac04753-107d-4586-89a5-d05578557c8c service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:08.107868 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac04753-107d-4586-89a5-d05578557c8c service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:08.108150 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac04753-107d-4586-89a5-d05578557c8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:08.117005 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac04753-107d-4586-89a5-d05578557c8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:08.118622 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac04753-107d-4586-89a5-d05578557c8c service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:08.123131 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:27:08.130157 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 690/1379] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:10.333823 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b44ab-4ac3-4634-b190-42ab8949a9fa None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:10.334368 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b44ab-4ac3-4634-b190-42ab8949a9fa None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:10.334368 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b44ab-4ac3-4634-b190-42ab8949a9fa None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:10.343168 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54d676-e28c-445c-b61d-996cff3e64d0 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:10.343710 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54d676-e28c-445c-b61d-996cff3e64d0 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:10.343710 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54d676-e28c-445c-b61d-996cff3e64d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:10.346472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 690/1380] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:27:10 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:10.364353 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-1a54d676-e28c-445c-b61d-996cff3e64d0 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:10.366191 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-1a54d676-e28c-445c-b61d-996cff3e64d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:10.378023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 691/1381] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:10.425721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b44ab-4ac3-4634-b190-42ab8949a9fa None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:10.445750 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b44ab-4ac3-4634-b190-42ab8949a9fa None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:10.457036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f5638-9c6a-43c5-82d7-3d4e3822d678 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:10.458091 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f5638-9c6a-43c5-82d7-3d4e3822d678 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:10.458538 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f5638-9c6a-43c5-82d7-3d4e3822d678 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:10.458823 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f5638-9c6a-43c5-82d7-3d4e3822d678 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:10.478025 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f5638-9c6a-43c5-82d7-3d4e3822d678 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:10.482510 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f5638-9c6a-43c5-82d7-3d4e3822d678 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:10.493524 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 691/1382] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:12.701269 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c76531-5eca-4e88-932b-973cd3409067 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:12.701836 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c76531-5eca-4e88-932b-973cd3409067 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:12.701836 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c76531-5eca-4e88-932b-973cd3409067 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:12.703227 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 692/1383] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:12.708510 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bada4-342f-46c6-8a19-d824f930c6a3 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:12.708938 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bada4-342f-46c6-8a19-d824f930c6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:12.709079 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bada4-342f-46c6-8a19-d824f930c6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:12.727793 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a65bada4-342f-46c6-8a19-d824f930c6a3 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:12.728989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a65bada4-342f-46c6-8a19-d824f930c6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:12.737415 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 692/1384] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:12.780886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c76531-5eca-4e88-932b-973cd3409067 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:12.795984 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c76531-5eca-4e88-932b-973cd3409067 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:12.802100 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d8b4f-f22f-40fc-bfe3-632c888cd850 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:12.802752 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d8b4f-f22f-40fc-bfe3-632c888cd850 service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:12.803047 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d8b4f-f22f-40fc-bfe3-632c888cd850 service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:12.803331 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d8b4f-f22f-40fc-bfe3-632c888cd850 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:12.813471 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d8b4f-f22f-40fc-bfe3-632c888cd850 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:12.815066 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d8b4f-f22f-40fc-bfe3-632c888cd850 service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:12.826373 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 693/1385] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:15.097280 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ddd2f-8490-4398-8046-724d6ff92eb4 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:15.097883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ddd2f-8490-4398-8046-724d6ff92eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:15.097883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ddd2f-8490-4398-8046-724d6ff92eb4 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:15.099286 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 693/1386] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:15.106125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676da32a-9afc-410b-9079-a2d4f2f66e47 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:15.106619 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676da32a-9afc-410b-9079-a2d4f2f66e47 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:15.106719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676da32a-9afc-410b-9079-a2d4f2f66e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:15.126198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-676da32a-9afc-410b-9079-a2d4f2f66e47 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:15.127461 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-676da32a-9afc-410b-9079-a2d4f2f66e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:15.136035 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 694/1387] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:27:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:15.183699 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ddd2f-8490-4398-8046-724d6ff92eb4 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:15.199778 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ddd2f-8490-4398-8046-724d6ff92eb4 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:15.205432 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1d4ae-26a7-42bf-8417-959a95107331 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:15.206117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1d4ae-26a7-42bf-8417-959a95107331 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:15.206418 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1d4ae-26a7-42bf-8417-959a95107331 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:15.206714 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1d4ae-26a7-42bf-8417-959a95107331 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:15.215772 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c1d4ae-26a7-42bf-8417-959a95107331 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:15.217480 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c1d4ae-26a7-42bf-8417-959a95107331 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:15.227795 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 694/1388] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:17.579041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d1bb3-cc01-426d-a7eb-c21171222f9c None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:17.580222 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d1bb3-cc01-426d-a7eb-c21171222f9c None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:17.580222 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d1bb3-cc01-426d-a7eb-c21171222f9c None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:17.581199 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 695/1389] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:17.586129 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb73217-8594-43a2-b792-aefeadccbdea None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:17.587180 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb73217-8594-43a2-b792-aefeadccbdea None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:17.587180 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb73217-8594-43a2-b792-aefeadccbdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:17.606952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0eb73217-8594-43a2-b792-aefeadccbdea None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:17.608166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0eb73217-8594-43a2-b792-aefeadccbdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:17.624367 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 695/1390] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:27:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:17.669902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d1bb3-cc01-426d-a7eb-c21171222f9c None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:17.689461 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d1bb3-cc01-426d-a7eb-c21171222f9c None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:17.695387 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4e69f-ab92-42d9-aca8-aec50247822c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:17.696017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4e69f-ab92-42d9-aca8-aec50247822c service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:17.696316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4e69f-ab92-42d9-aca8-aec50247822c service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:17.696605 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4e69f-ab92-42d9-aca8-aec50247822c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:17.706485 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4e69f-ab92-42d9-aca8-aec50247822c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:17.708086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4e69f-ab92-42d9-aca8-aec50247822c service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:17.719154 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 696/1391] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:20.044889 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b8bc4-68a0-4463-846d-fd1f9d0c39bd None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:20.045563 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b8bc4-68a0-4463-846d-fd1f9d0c39bd None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:20.045563 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b8bc4-68a0-4463-846d-fd1f9d0c39bd None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:20.046870 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 696/1392] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:20.053637 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0c72c-c99a-436c-904e-fa8e117a64af None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:20.054140 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0c72c-c99a-436c-904e-fa8e117a64af None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:20.054242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0c72c-c99a-436c-904e-fa8e117a64af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:20.074175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-19d0c72c-c99a-436c-904e-fa8e117a64af None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:20.075502 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-19d0c72c-c99a-436c-904e-fa8e117a64af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:20.084485 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 697/1393] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:20.140193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b8bc4-68a0-4463-846d-fd1f9d0c39bd None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:20.159436 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b8bc4-68a0-4463-846d-fd1f9d0c39bd None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:20.166243 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b6dc4-4dac-4868-a312-7b67abdef8b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:20.167086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b6dc4-4dac-4868-a312-7b67abdef8b0 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:20.167446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b6dc4-4dac-4868-a312-7b67abdef8b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:20.167805 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b6dc4-4dac-4868-a312-7b67abdef8b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:20.178695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b6dc4-4dac-4868-a312-7b67abdef8b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:20.180810 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b6dc4-4dac-4868-a312-7b67abdef8b0 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:20.192942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 697/1394] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:22.385128 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d964e-9074-43de-b349-cd2b85ded106 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:22.385705 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d964e-9074-43de-b349-cd2b85ded106 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:22.385872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d964e-9074-43de-b349-cd2b85ded106 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:22.387750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 698/1395] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:27:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:22.390454 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7236b-c367-4dee-a78f-0c0dabe05557 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:22.390899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7236b-c367-4dee-a78f-0c0dabe05557 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:22.391003 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7236b-c367-4dee-a78f-0c0dabe05557 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:22.392401 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 698/1396] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:22.392928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ffe888-60f7-4c5f-933a-eb79a121d848 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:22.393260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ffe888-60f7-4c5f-933a-eb79a121d848 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:22.393549 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ffe888-60f7-4c5f-933a-eb79a121d848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:22.397359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406db5df-ea38-4f98-9880-2290cb02ae73 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:22.397646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406db5df-ea38-4f98-9880-2290cb02ae73 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:22.397987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406db5df-ea38-4f98-9880-2290cb02ae73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:22.404169 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=52 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:27:22.412932 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-98ffe888-60f7-4c5f-933a-eb79a121d848 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:22.414188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-98ffe888-60f7-4c5f-933a-eb79a121d848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:22.416839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-406db5df-ea38-4f98-9880-2290cb02ae73 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:22.418579 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-406db5df-ea38-4f98-9880-2290cb02ae73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:22.422771 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 699/1397] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:27:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:27:22.426335 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 699/1398] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:27:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:27:22.469727 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ffe888-60f7-4c5f-933a-eb79a121d848 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:22.480568 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a4885-170c-4bc3-9fd5-7356ece195f2 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:22.481108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a4885-170c-4bc3-9fd5-7356ece195f2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:22.481416 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a4885-170c-4bc3-9fd5-7356ece195f2 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:22.482582 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ffe888-60f7-4c5f-933a-eb79a121d848 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:22.482826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 700/1399] 10.222.0.40 () {54 vars in 962 bytes} [Thu Jun 16 17:27:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:27:22.487939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff0275-e465-42a2-a4e5-990cb6e62f01 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:22.488240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff0275-e465-42a2-a4e5-990cb6e62f01 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:22.488437 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c399ca0-4f53-47f6-8d05-8f2b9f94a51f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:22.488637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff0275-e465-42a2-a4e5-990cb6e62f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:22.488967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c399ca0-4f53-47f6-8d05-8f2b9f94a51f service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:22.491278 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c399ca0-4f53-47f6-8d05-8f2b9f94a51f service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:22.491278 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c399ca0-4f53-47f6-8d05-8f2b9f94a51f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:22.501381 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c399ca0-4f53-47f6-8d05-8f2b9f94a51f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:22.503576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c399ca0-4f53-47f6-8d05-8f2b9f94a51f service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:22.513968 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 700/1400] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:22.541467 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-c2ff0275-e465-42a2-a4e5-990cb6e62f01 None None] MFA Rules not processed for user `327b347d6295422c99ce7b44eab00467`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:22.543556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c2ff0275-e465-42a2-a4e5-990cb6e62f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:22.608877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 701/1401] 10.222.0.40 () {58 vars in 1059 bytes} [Thu Jun 16 17:27:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:27:22.617124 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c399ca0-4f53-47f6-8d05-8f2b9f94a51f service nova] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:22.626822 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c399ca0-4f53-47f6-8d05-8f2b9f94a51f service nova] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:22.630779 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a7b9e-e153-4859-a76a-232266eddb34 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327b347d6295422c99ce7b44eab00467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:22.631435 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a7b9e-e153-4859-a76a-232266eddb34 service glance] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:22.631725 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a7b9e-e153-4859-a76a-232266eddb34 service glance] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:22.632006 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a7b9e-e153-4859-a76a-232266eddb34 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:22.642761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a7b9e-e153-4859-a76a-232266eddb34 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:22.644215 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a7b9e-e153-4859-a76a-232266eddb34 service glance] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:22.655587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 701/1402] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:23.376534 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374be84-485e-4355-94b8-7f455d7635ae None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:23.376832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374be84-485e-4355-94b8-7f455d7635ae None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:23.377120 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374be84-485e-4355-94b8-7f455d7635ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:23.428390 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d374be84-485e-4355-94b8-7f455d7635ae None None] MFA Rules not processed for user `cfc2f31f654e486d8505d80aad17d17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:23.430457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d374be84-485e-4355-94b8-7f455d7635ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:23.498135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 702/1403] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 17:27:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:23.545182 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3d5f2-439e-47c7-bbb6-12e69fde8b26 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:23.545690 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3d5f2-439e-47c7-bbb6-12e69fde8b26 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:23.545802 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3d5f2-439e-47c7-bbb6-12e69fde8b26 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:23.547131 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 702/1404] 10.222.0.40 () {54 vars in 942 bytes} [Thu Jun 16 17:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:23.553733 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbf2cd-e750-4da9-9d4b-47de8d28c5f2 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:23.554066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbf2cd-e750-4da9-9d4b-47de8d28c5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:23.554413 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbf2cd-e750-4da9-9d4b-47de8d28c5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:23.603202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-49dbf2cd-e750-4da9-9d4b-47de8d28c5f2 None None] MFA Rules not processed for user `b8a37d1201d54afdad9b228ef4f2d787`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:23.604407 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-49dbf2cd-e750-4da9-9d4b-47de8d28c5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:23.618340 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:27:23.681114 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 703/1405] 10.222.0.40 () {58 vars in 1039 bytes} [Thu Jun 16 17:27:23 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:23.690468 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3d5f2-439e-47c7-bbb6-12e69fde8b26 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:23.704512 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3d5f2-439e-47c7-bbb6-12e69fde8b26 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:23.709505 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d6b00-8968-469c-8bf0-ac00f80f159e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a37d1201d54afdad9b228ef4f2d787', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:23.710315 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d6b00-8968-469c-8bf0-ac00f80f159e service swift] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:23.710607 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d6b00-8968-469c-8bf0-ac00f80f159e service swift] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:23.710888 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d6b00-8968-469c-8bf0-ac00f80f159e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:23.720393 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d6b00-8968-469c-8bf0-ac00f80f159e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:23.722058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d6b00-8968-469c-8bf0-ac00f80f159e service swift] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:23.730951 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 703/1406] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 17:27:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:27:24.703472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeabcb-5bbd-4420-b537-f81666fd60e8 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:24.704062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeabcb-5bbd-4420-b537-f81666fd60e8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:24.704062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeabcb-5bbd-4420-b537-f81666fd60e8 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:24.705469 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 704/1407] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:24.710909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc676f97-6031-44cb-a047-fcac2aa9b86e None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:24.711367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc676f97-6031-44cb-a047-fcac2aa9b86e None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:24.711489 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc676f97-6031-44cb-a047-fcac2aa9b86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:24.730837 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-fc676f97-6031-44cb-a047-fcac2aa9b86e None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:24.732064 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-fc676f97-6031-44cb-a047-fcac2aa9b86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:24.740376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 704/1408] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:24.786740 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eeabcb-5bbd-4420-b537-f81666fd60e8 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:24.802604 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eeabcb-5bbd-4420-b537-f81666fd60e8 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:24.809008 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01f4d3-c6de-456a-a05d-e3d6f887a5ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:24.809834 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01f4d3-c6de-456a-a05d-e3d6f887a5ec service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:24.810117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01f4d3-c6de-456a-a05d-e3d6f887a5ec service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:24.810400 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01f4d3-c6de-456a-a05d-e3d6f887a5ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:24.819605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01f4d3-c6de-456a-a05d-e3d6f887a5ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:24.821376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01f4d3-c6de-456a-a05d-e3d6f887a5ec service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:24.832082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 705/1409] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:34.725401 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=46 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:27:34.828065 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=30 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:27:46.737653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1caea-2bc1-47c5-a07b-03e6f512ec9c None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:46.738261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1caea-2bc1-47c5-a07b-03e6f512ec9c None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:46.738261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1caea-2bc1-47c5-a07b-03e6f512ec9c None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:46.739584 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 705/1410] 10.222.0.40 () {54 vars in 918 bytes} [Thu Jun 16 17:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:46.745004 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4a6e7-3d0e-4ce0-927b-9c5696f4361c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:46.745491 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4a6e7-3d0e-4ce0-927b-9c5696f4361c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:46.745731 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4a6e7-3d0e-4ce0-927b-9c5696f4361c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:46.767269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-01b4a6e7-3d0e-4ce0-927b-9c5696f4361c None None] MFA Rules not processed for user `75f0586a8c1941448d750918183f6dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:46.768620 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-01b4a6e7-3d0e-4ce0-927b-9c5696f4361c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:46.778349 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 706/1411] 10.222.0.40 () {58 vars in 1015 bytes} [Thu Jun 16 17:27:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:46.792562 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ed47f-a174-4751-80d5-aa9d735c3ab5 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:46.792861 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ed47f-a174-4751-80d5-aa9d735c3ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:46.793173 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ed47f-a174-4751-80d5-aa9d735c3ab5 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:46.794687 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 706/1412] 10.222.0.40 () {54 vars in 918 bytes} [Thu Jun 16 17:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:46.799306 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9643d0-371d-40a7-b3b8-bc8fd1923adc None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:46.799605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9643d0-371d-40a7-b3b8-bc8fd1923adc None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:46.799891 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9643d0-371d-40a7-b3b8-bc8fd1923adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:46.822609 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1b9643d0-371d-40a7-b3b8-bc8fd1923adc None None] MFA Rules not processed for user `689a31e26c6b454a91c3fd79678b4d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:46.824888 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1b9643d0-371d-40a7-b3b8-bc8fd1923adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:46.836462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 707/1413] 10.222.0.40 () {58 vars in 1015 bytes} [Thu Jun 16 17:27:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:49.492328 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa1c9a-ccc6-419f-a8ec-2de8ff9a936f None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:49.493131 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa1c9a-ccc6-419f-a8ec-2de8ff9a936f None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:49.493131 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa1c9a-ccc6-419f-a8ec-2de8ff9a936f None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:49.494457 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:49.500316 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277dfde6-98f2-45fd-8000-fb1003c0f25f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:49.500781 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277dfde6-98f2-45fd-8000-fb1003c0f25f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:49.500897 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277dfde6-98f2-45fd-8000-fb1003c0f25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:49.520152 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-277dfde6-98f2-45fd-8000-fb1003c0f25f None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:49.521407 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-277dfde6-98f2-45fd-8000-fb1003c0f25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:49.529959 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:27:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:49.573773 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa1c9a-ccc6-419f-a8ec-2de8ff9a936f None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:49.588941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa1c9a-ccc6-419f-a8ec-2de8ff9a936f None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:49.595356 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d935f3-bff4-4bae-a995-e58fe2d67a71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:49.596010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d935f3-bff4-4bae-a995-e58fe2d67a71 service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:49.596304 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d935f3-bff4-4bae-a995-e58fe2d67a71 service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:49.596590 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d935f3-bff4-4bae-a995-e58fe2d67a71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:49.605050 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d935f3-bff4-4bae-a995-e58fe2d67a71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:49.606998 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d935f3-bff4-4bae-a995-e58fe2d67a71 service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:49.616670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 708/1416] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:50.594460 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d668219-28d4-4cda-8899-3859bf8b08f3 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:50.595068 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d668219-28d4-4cda-8899-3859bf8b08f3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:50.595068 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d668219-28d4-4cda-8899-3859bf8b08f3 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:50.596539 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:50.601741 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c030f92-89df-4fb0-8261-db2b4a15fcb8 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:50.602161 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c030f92-89df-4fb0-8261-db2b4a15fcb8 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:50.602161 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c030f92-89df-4fb0-8261-db2b4a15fcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:50.620806 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-0c030f92-89df-4fb0-8261-db2b4a15fcb8 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:50.622082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0c030f92-89df-4fb0-8261-db2b4a15fcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:50.629882 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:50.781713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d668219-28d4-4cda-8899-3859bf8b08f3 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:50.797592 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d668219-28d4-4cda-8899-3859bf8b08f3 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:50.803075 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0534c-dd59-4a2e-abd7-6254f4343811 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:50.803729 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0534c-dd59-4a2e-abd7-6254f4343811 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:50.804014 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0534c-dd59-4a2e-abd7-6254f4343811 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:50.804297 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0534c-dd59-4a2e-abd7-6254f4343811 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:50.816296 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0534c-dd59-4a2e-abd7-6254f4343811 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:50.817927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0534c-dd59-4a2e-abd7-6254f4343811 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:50.828439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 710/1419] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:51.773861 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a90498-1a8d-4608-90c3-c431fc547096 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:51.774534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a90498-1a8d-4608-90c3-c431fc547096 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:51.774534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a90498-1a8d-4608-90c3-c431fc547096 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:51.775938 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:51.781813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18308246-71e8-48d1-9d9e-c45c1eded655 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:51.782261 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18308246-71e8-48d1-9d9e-c45c1eded655 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:51.782502 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18308246-71e8-48d1-9d9e-c45c1eded655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:51.804747 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-18308246-71e8-48d1-9d9e-c45c1eded655 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:51.806397 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-18308246-71e8-48d1-9d9e-c45c1eded655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:51.815610 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:27:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:51.994700 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a90498-1a8d-4608-90c3-c431fc547096 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:52.009303 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a90498-1a8d-4608-90c3-c431fc547096 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:52.015036 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98889d3a-4f38-4c5a-9b44-f1e6e0946f91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:52.015684 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98889d3a-4f38-4c5a-9b44-f1e6e0946f91 service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:52.015981 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98889d3a-4f38-4c5a-9b44-f1e6e0946f91 service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:52.016268 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98889d3a-4f38-4c5a-9b44-f1e6e0946f91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:52.024877 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98889d3a-4f38-4c5a-9b44-f1e6e0946f91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:52.026447 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98889d3a-4f38-4c5a-9b44-f1e6e0946f91 service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:52.036876 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 711/1422] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:52.971332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3fefe-e0b8-4a32-ad64-bcab2d16c51e None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:52.971893 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3fefe-e0b8-4a32-ad64-bcab2d16c51e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:52.972018 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3fefe-e0b8-4a32-ad64-bcab2d16c51e None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:52.973523 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:52.978726 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e83ad-6e2f-456e-8f92-d90de9299fce None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:52.979135 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e83ad-6e2f-456e-8f92-d90de9299fce None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:52.979422 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e83ad-6e2f-456e-8f92-d90de9299fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:53.000317 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-ff5e83ad-6e2f-456e-8f92-d90de9299fce None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:53.001678 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-ff5e83ad-6e2f-456e-8f92-d90de9299fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:53.010383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:27:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:53.159522 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af3fefe-e0b8-4a32-ad64-bcab2d16c51e None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:53.174884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af3fefe-e0b8-4a32-ad64-bcab2d16c51e None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:53.180290 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd63bd-80a7-4b79-93c1-2d705496701f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:53.180935 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd63bd-80a7-4b79-93c1-2d705496701f service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:53.181256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd63bd-80a7-4b79-93c1-2d705496701f service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:53.181549 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd63bd-80a7-4b79-93c1-2d705496701f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:53.191012 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd63bd-80a7-4b79-93c1-2d705496701f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:53.192605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd63bd-80a7-4b79-93c1-2d705496701f service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:53.202683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 713/1425] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:54.133186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a961f-c48c-4bd5-b1e3-efaeed910631 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:54.133856 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a961f-c48c-4bd5-b1e3-efaeed910631 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:54.133856 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a961f-c48c-4bd5-b1e3-efaeed910631 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:54.135172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:54.140218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49240d47-7682-4bd4-874b-851839bdf0d5 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:54.140702 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49240d47-7682-4bd4-874b-851839bdf0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:54.140806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49240d47-7682-4bd4-874b-851839bdf0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:54.159155 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-49240d47-7682-4bd4-874b-851839bdf0d5 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:54.160369 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-49240d47-7682-4bd4-874b-851839bdf0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:54.168876 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:27:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:54.342280 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a961f-c48c-4bd5-b1e3-efaeed910631 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:54.357461 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a961f-c48c-4bd5-b1e3-efaeed910631 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:54.363099 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb8717-a793-49ed-8f3e-499d4fbedd40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:54.363743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb8717-a793-49ed-8f3e-499d4fbedd40 service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:54.364037 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb8717-a793-49ed-8f3e-499d4fbedd40 service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:54.364324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb8717-a793-49ed-8f3e-499d4fbedd40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:54.373631 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb8717-a793-49ed-8f3e-499d4fbedd40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:54.375220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb8717-a793-49ed-8f3e-499d4fbedd40 service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:54.384538 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 714/1428] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:55.249689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0276d-f0a5-47bc-bd05-03ac8769b6c2 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:55.250246 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0276d-f0a5-47bc-bd05-03ac8769b6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:55.250375 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0276d-f0a5-47bc-bd05-03ac8769b6c2 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:55.251775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:27:55.256710 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26f794-d35b-42bb-80e8-21ffa039f770 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:55.257200 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26f794-d35b-42bb-80e8-21ffa039f770 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:55.257435 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26f794-d35b-42bb-80e8-21ffa039f770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:55.275872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-3c26f794-d35b-42bb-80e8-21ffa039f770 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:55.277082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-3c26f794-d35b-42bb-80e8-21ffa039f770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:55.285540 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:55.437645 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0276d-f0a5-47bc-bd05-03ac8769b6c2 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:55.452908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0276d-f0a5-47bc-bd05-03ac8769b6c2 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:55.458303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57b7b9-d268-4429-94bf-af94bc3d766f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:55.458950 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57b7b9-d268-4429-94bf-af94bc3d766f service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:55.459239 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57b7b9-d268-4429-94bf-af94bc3d766f service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:55.459527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57b7b9-d268-4429-94bf-af94bc3d766f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:55.468824 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57b7b9-d268-4429-94bf-af94bc3d766f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:55.470452 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57b7b9-d268-4429-94bf-af94bc3d766f service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:55.479946 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 716/1431] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:56.336877 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766845f2-4883-41c6-b523-f03694291187 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:56.337562 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766845f2-4883-41c6-b523-f03694291187 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:56.337562 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766845f2-4883-41c6-b523-f03694291187 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:56.338847 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:56.344427 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24136151-7d89-42ee-8ec3-7e94b9d93355 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:56.344857 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24136151-7d89-42ee-8ec3-7e94b9d93355 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:56.345019 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24136151-7d89-42ee-8ec3-7e94b9d93355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:56.363305 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-24136151-7d89-42ee-8ec3-7e94b9d93355 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:56.364493 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-24136151-7d89-42ee-8ec3-7e94b9d93355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:56.372500 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:27:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:56.524243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766845f2-4883-41c6-b523-f03694291187 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:56.539177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766845f2-4883-41c6-b523-f03694291187 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:56.545285 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446a773-797f-4921-833b-3278249a85fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:56.545934 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446a773-797f-4921-833b-3278249a85fc service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:56.546231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446a773-797f-4921-833b-3278249a85fc service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:56.546516 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446a773-797f-4921-833b-3278249a85fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:56.554793 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446a773-797f-4921-833b-3278249a85fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:56.556374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446a773-797f-4921-833b-3278249a85fc service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:56.566237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 717/1434] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:57.418679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6f641-6ca7-4a5e-90a2-6f4ea5fa052f None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:57.419856 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6f641-6ca7-4a5e-90a2-6f4ea5fa052f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:57.419856 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6f641-6ca7-4a5e-90a2-6f4ea5fa052f None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:57.421268 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:27:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:57.426113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2d92b-6bf9-4836-96c8-9128c1a244fb None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:57.426533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2d92b-6bf9-4836-96c8-9128c1a244fb None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:57.426698 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2d92b-6bf9-4836-96c8-9128c1a244fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:57.445327 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-01e2d92b-6bf9-4836-96c8-9128c1a244fb None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:57.446537 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-01e2d92b-6bf9-4836-96c8-9128c1a244fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:57.454391 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:57.603136 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6f641-6ca7-4a5e-90a2-6f4ea5fa052f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:57.620339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6f641-6ca7-4a5e-90a2-6f4ea5fa052f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:57.626793 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041fd6c-06da-4618-ae4e-1260df07ba07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:57.627450 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041fd6c-06da-4618-ae4e-1260df07ba07 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:57.627738 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041fd6c-06da-4618-ae4e-1260df07ba07 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:57.628018 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041fd6c-06da-4618-ae4e-1260df07ba07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:57.640069 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d041fd6c-06da-4618-ae4e-1260df07ba07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:57.641846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d041fd6c-06da-4618-ae4e-1260df07ba07 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:57.652122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 719/1437] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:58.505845 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764dd92-3093-4008-a5cd-bad0c436ca54 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:58.506451 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764dd92-3093-4008-a5cd-bad0c436ca54 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:58.506451 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764dd92-3093-4008-a5cd-bad0c436ca54 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:58.507792 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:27:58.512743 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5fbf1-6c1b-41d8-9d5f-7a1cc3651e18 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:58.513725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5fbf1-6c1b-41d8-9d5f-7a1cc3651e18 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:58.513959 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5fbf1-6c1b-41d8-9d5f-7a1cc3651e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:58.531708 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-cfc5fbf1-6c1b-41d8-9d5f-7a1cc3651e18 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:58.532919 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-cfc5fbf1-6c1b-41d8-9d5f-7a1cc3651e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:58.540714 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:58.697680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f764dd92-3093-4008-a5cd-bad0c436ca54 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:58.712613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f764dd92-3093-4008-a5cd-bad0c436ca54 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:58.718231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ebf4d-d8d7-4871-9326-6f9b22266046 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:58.718905 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ebf4d-d8d7-4871-9326-6f9b22266046 service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:58.719197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ebf4d-d8d7-4871-9326-6f9b22266046 service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:58.719479 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ebf4d-d8d7-4871-9326-6f9b22266046 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:58.727618 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ebf4d-d8d7-4871-9326-6f9b22266046 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:58.729227 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ebf4d-d8d7-4871-9326-6f9b22266046 service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:58.738648 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 720/1440] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:00.749002 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdbf67-eea6-492f-be2d-72173f104301 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:00.749614 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdbf67-eea6-492f-be2d-72173f104301 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:00.749614 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdbf67-eea6-492f-be2d-72173f104301 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:00.750999 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:28:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:28:00.756351 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06607c-a379-42f4-9110-845f0ba5127a None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:00.756780 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06607c-a379-42f4-9110-845f0ba5127a None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:00.756975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06607c-a379-42f4-9110-845f0ba5127a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:00.774610 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-1e06607c-a379-42f4-9110-845f0ba5127a None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:00.775811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-1e06607c-a379-42f4-9110-845f0ba5127a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:00.784537 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:28:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:00.822644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdbf67-eea6-492f-be2d-72173f104301 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:00.833498 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdbf67-eea6-492f-be2d-72173f104301 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:00.836967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53532258-d8b7-49fd-a89e-2f17c718541e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:00.837653 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53532258-d8b7-49fd-a89e-2f17c718541e service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:00.837943 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53532258-d8b7-49fd-a89e-2f17c718541e service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:00.838227 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53532258-d8b7-49fd-a89e-2f17c718541e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:00.848424 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53532258-d8b7-49fd-a89e-2f17c718541e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:00.849916 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53532258-d8b7-49fd-a89e-2f17c718541e service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:00.860765 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 722/1443] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:03.213789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667df0fe-f22c-41c5-ba73-32f2382c195f None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:03.214370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667df0fe-f22c-41c5-ba73-32f2382c195f None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:03.214370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667df0fe-f22c-41c5-ba73-32f2382c195f None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:03.216054 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:28:03.221293 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a5aea-f2f2-4695-91ba-908508ba4d8e None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:03.221763 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a5aea-f2f2-4695-91ba-908508ba4d8e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:03.221914 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a5aea-f2f2-4695-91ba-908508ba4d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:03.240492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0d7a5aea-f2f2-4695-91ba-908508ba4d8e None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:03.241741 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0d7a5aea-f2f2-4695-91ba-908508ba4d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:03.250412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:03.289237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667df0fe-f22c-41c5-ba73-32f2382c195f None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:03.299950 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667df0fe-f22c-41c5-ba73-32f2382c195f None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:03.303464 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e2c6a-1266-4bb9-933c-5d7fad0a6dfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:03.304099 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e2c6a-1266-4bb9-933c-5d7fad0a6dfc service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:03.304390 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e2c6a-1266-4bb9-933c-5d7fad0a6dfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:03.304676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e2c6a-1266-4bb9-933c-5d7fad0a6dfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:03.314156 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e2c6a-1266-4bb9-933c-5d7fad0a6dfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:03.315579 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e2c6a-1266-4bb9-933c-5d7fad0a6dfc service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:03.325170 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 723/1446] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:05.854910 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df24be-7dbf-4db0-8b7f-5aa4eeda06a7 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:05.855470 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df24be-7dbf-4db0-8b7f-5aa4eeda06a7 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:05.855592 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df24be-7dbf-4db0-8b7f-5aa4eeda06a7 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:05.856918 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:28:05.862765 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb68537-3c82-46ee-9d23-6e3dc6ecfa51 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:05.863218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb68537-3c82-46ee-9d23-6e3dc6ecfa51 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:05.863326 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb68537-3c82-46ee-9d23-6e3dc6ecfa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:05.882302 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-2bb68537-3c82-46ee-9d23-6e3dc6ecfa51 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:05.883511 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-2bb68537-3c82-46ee-9d23-6e3dc6ecfa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:05.893405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:05.931993 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df24be-7dbf-4db0-8b7f-5aa4eeda06a7 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:05.944028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df24be-7dbf-4db0-8b7f-5aa4eeda06a7 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:05.947593 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbbd07-5b7d-4d3c-93e3-1eaa2c8655f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:05.948267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbbd07-5b7d-4d3c-93e3-1eaa2c8655f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:05.948558 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbbd07-5b7d-4d3c-93e3-1eaa2c8655f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:05.948847 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbbd07-5b7d-4d3c-93e3-1eaa2c8655f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:05.957813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbbd07-5b7d-4d3c-93e3-1eaa2c8655f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:05.959254 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbbd07-5b7d-4d3c-93e3-1eaa2c8655f9 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:05.968488 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 725/1449] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:08.267343 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599021f6-7126-4877-b0cb-739e0ce9081d None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:08.267876 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599021f6-7126-4877-b0cb-739e0ce9081d None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:08.267971 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599021f6-7126-4877-b0cb-739e0ce9081d None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:08.269313 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:28:08.274836 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9dd7a3-4ecb-4f9a-8c27-d3da18d37042 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:08.275275 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9dd7a3-4ecb-4f9a-8c27-d3da18d37042 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:08.275456 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9dd7a3-4ecb-4f9a-8c27-d3da18d37042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:08.292888 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0b9dd7a3-4ecb-4f9a-8c27-d3da18d37042 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:08.294124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0b9dd7a3-4ecb-4f9a-8c27-d3da18d37042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:08.301915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:28:08.340991 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599021f6-7126-4877-b0cb-739e0ce9081d None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:08.351863 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599021f6-7126-4877-b0cb-739e0ce9081d None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:08.355360 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84eb82c-1b00-4aab-8ef3-2e3fae0f2613 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:08.355997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84eb82c-1b00-4aab-8ef3-2e3fae0f2613 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:08.356289 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84eb82c-1b00-4aab-8ef3-2e3fae0f2613 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:08.356591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84eb82c-1b00-4aab-8ef3-2e3fae0f2613 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:08.365537 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84eb82c-1b00-4aab-8ef3-2e3fae0f2613 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:08.366968 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84eb82c-1b00-4aab-8ef3-2e3fae0f2613 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:08.377302 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 726/1452] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:10.579194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef781ce4-a3ef-4c54-afac-485b28d42365 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:10.579745 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef781ce4-a3ef-4c54-afac-485b28d42365 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:10.579846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef781ce4-a3ef-4c54-afac-485b28d42365 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:10.581230 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:28:10.586335 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8986b8-4c30-45ef-ad65-32b3031fb65b None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:10.586786 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8986b8-4c30-45ef-ad65-32b3031fb65b None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:10.586918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8986b8-4c30-45ef-ad65-32b3031fb65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:10.605017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-5c8986b8-4c30-45ef-ad65-32b3031fb65b None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:10.606249 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-5c8986b8-4c30-45ef-ad65-32b3031fb65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:10.614912 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:28:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:10.653241 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef781ce4-a3ef-4c54-afac-485b28d42365 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:10.664849 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef781ce4-a3ef-4c54-afac-485b28d42365 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:10.668403 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524a75f-df4a-4a30-94a4-c067846de692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:10.669071 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524a75f-df4a-4a30-94a4-c067846de692 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:10.669398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524a75f-df4a-4a30-94a4-c067846de692 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:10.669681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524a75f-df4a-4a30-94a4-c067846de692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:10.679289 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524a75f-df4a-4a30-94a4-c067846de692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:10.680723 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524a75f-df4a-4a30-94a4-c067846de692 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:10.690034 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 728/1455] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:12.942759 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f161d9-7705-4d44-b2d7-f7dc09353403 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:12.943365 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f161d9-7705-4d44-b2d7-f7dc09353403 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:12.943365 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f161d9-7705-4d44-b2d7-f7dc09353403 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:12.944705 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:28:12.950083 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dee7d7-4be9-490b-9599-224e4593e953 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:12.950523 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dee7d7-4be9-490b-9599-224e4593e953 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:12.950675 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dee7d7-4be9-490b-9599-224e4593e953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:12.969002 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-40dee7d7-4be9-490b-9599-224e4593e953 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:12.970230 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-40dee7d7-4be9-490b-9599-224e4593e953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:12.977929 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:28:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:13.015668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f161d9-7705-4d44-b2d7-f7dc09353403 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:13.026298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f161d9-7705-4d44-b2d7-f7dc09353403 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:13.030931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532e282-083f-43ac-ac13-3fefceda3f96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:13.031577 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532e282-083f-43ac-ac13-3fefceda3f96 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:13.031871 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532e282-083f-43ac-ac13-3fefceda3f96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:13.032159 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532e282-083f-43ac-ac13-3fefceda3f96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:13.041057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532e282-083f-43ac-ac13-3fefceda3f96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:13.042531 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532e282-083f-43ac-ac13-3fefceda3f96 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:13.052966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 729/1458] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:15.218328 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b66558-4103-4a21-90fc-b9a43bc536e1 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:15.218945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b66558-4103-4a21-90fc-b9a43bc536e1 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:15.218945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b66558-4103-4a21-90fc-b9a43bc536e1 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:15.220304 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:28:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:28:15.225627 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c69090-3f4b-4e92-8fe4-7d60b726a226 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:15.226067 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c69090-3f4b-4e92-8fe4-7d60b726a226 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:15.226203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c69090-3f4b-4e92-8fe4-7d60b726a226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:15.232333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=70 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:15.246709 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-a6c69090-3f4b-4e92-8fe4-7d60b726a226 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:15.247946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-a6c69090-3f4b-4e92-8fe4-7d60b726a226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:15.255917 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:15.292947 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b66558-4103-4a21-90fc-b9a43bc536e1 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:15.299035 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=73 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:15.305490 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b66558-4103-4a21-90fc-b9a43bc536e1 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:15.308943 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96184094-a156-468d-9d95-e51704a0eaa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:15.309625 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96184094-a156-468d-9d95-e51704a0eaa3 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:15.309913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96184094-a156-468d-9d95-e51704a0eaa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:15.310200 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96184094-a156-468d-9d95-e51704a0eaa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:15.319352 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96184094-a156-468d-9d95-e51704a0eaa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:15.320783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96184094-a156-468d-9d95-e51704a0eaa3 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:15.330182 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 731/1461] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:17.520674 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6a054-667d-413b-9f38-0198bd0b8164 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:17.521248 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6a054-667d-413b-9f38-0198bd0b8164 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:17.521248 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6a054-667d-413b-9f38-0198bd0b8164 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:17.522585 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:28:17.527789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80108984-eb9a-4cbf-bbed-9f960efcb820 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:17.528191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80108984-eb9a-4cbf-bbed-9f960efcb820 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:17.528387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80108984-eb9a-4cbf-bbed-9f960efcb820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:17.546682 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-80108984-eb9a-4cbf-bbed-9f960efcb820 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:17.547885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-80108984-eb9a-4cbf-bbed-9f960efcb820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:17.555612 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:17.593544 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6a054-667d-413b-9f38-0198bd0b8164 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:17.604058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6a054-667d-413b-9f38-0198bd0b8164 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:17.607573 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591111dc-0b3b-4b78-a821-051586e083c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:17.608715 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591111dc-0b3b-4b78-a821-051586e083c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:17.609001 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591111dc-0b3b-4b78-a821-051586e083c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:17.609316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591111dc-0b3b-4b78-a821-051586e083c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:17.618190 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591111dc-0b3b-4b78-a821-051586e083c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:17.619868 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591111dc-0b3b-4b78-a821-051586e083c6 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:17.629748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 732/1464] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:19.773922 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbb646-316a-4570-b0d2-2c6d45e2613f None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:19.774500 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbb646-316a-4570-b0d2-2c6d45e2613f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:19.774500 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbb646-316a-4570-b0d2-2c6d45e2613f None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:19.775800 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:28:19.781724 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f5e6a-cc9d-498d-9124-18168739a0c9 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:19.782214 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f5e6a-cc9d-498d-9124-18168739a0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:19.782323 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f5e6a-cc9d-498d-9124-18168739a0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:19.800389 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-684f5e6a-cc9d-498d-9124-18168739a0c9 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:19.801607 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-684f5e6a-cc9d-498d-9124-18168739a0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:19.809303 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:19.845424 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbb646-316a-4570-b0d2-2c6d45e2613f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:19.856078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbb646-316a-4570-b0d2-2c6d45e2613f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:19.861240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9ac1d-6ea7-467a-bc31-f70802ae17aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:19.861887 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9ac1d-6ea7-467a-bc31-f70802ae17aa service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:19.862208 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9ac1d-6ea7-467a-bc31-f70802ae17aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:19.862486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9ac1d-6ea7-467a-bc31-f70802ae17aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:19.872002 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9ac1d-6ea7-467a-bc31-f70802ae17aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:19.873588 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9ac1d-6ea7-467a-bc31-f70802ae17aa service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:19.882873 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 734/1467] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:22.068970 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f46b62-fdea-4086-b366-ec8608fbcf8b None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:22.069557 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f46b62-fdea-4086-b366-ec8608fbcf8b None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:22.069557 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f46b62-fdea-4086-b366-ec8608fbcf8b None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:22.070936 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:28:22.076060 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630b478-8ba4-48cd-8915-0c723165b1aa None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:22.076510 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630b478-8ba4-48cd-8915-0c723165b1aa None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:22.076682 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630b478-8ba4-48cd-8915-0c723165b1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:22.095745 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0630b478-8ba4-48cd-8915-0c723165b1aa None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:22.096964 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0630b478-8ba4-48cd-8915-0c723165b1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:22.104903 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:28:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:22.142865 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f46b62-fdea-4086-b366-ec8608fbcf8b None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:22.153497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f46b62-fdea-4086-b366-ec8608fbcf8b None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:22.157849 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ecfa2-b1d6-43ee-9d4b-8a467a65266a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:22.158487 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ecfa2-b1d6-43ee-9d4b-8a467a65266a service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:22.158775 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ecfa2-b1d6-43ee-9d4b-8a467a65266a service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:22.159054 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ecfa2-b1d6-43ee-9d4b-8a467a65266a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:22.167750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ecfa2-b1d6-43ee-9d4b-8a467a65266a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:22.169209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ecfa2-b1d6-43ee-9d4b-8a467a65266a service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:22.178467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 735/1470] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:32.089357 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=17 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:32.174438 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=17 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:42.468764 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4db2b2-a74f-4b00-a16a-7f89f7035179 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:42.469359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4db2b2-a74f-4b00-a16a-7f89f7035179 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:42.469359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4db2b2-a74f-4b00-a16a-7f89f7035179 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:42.470670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 736/1471] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:28:42.475818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29eec0c-6b7a-498c-b468-882de1b104b0 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:42.476263 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29eec0c-6b7a-498c-b468-882de1b104b0 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:42.476383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29eec0c-6b7a-498c-b468-882de1b104b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:42.494674 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-a29eec0c-6b7a-498c-b468-882de1b104b0 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:42.495876 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-a29eec0c-6b7a-498c-b468-882de1b104b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:42.504944 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 736/1472] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:42.551604 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6f2d5-3989-490f-946b-e34759333a96 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:42.551901 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6f2d5-3989-490f-946b-e34759333a96 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:42.552183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6f2d5-3989-490f-946b-e34759333a96 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:42.553592 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 737/1473] 10.222.0.40 () {54 vars in 962 bytes} [Thu Jun 16 17:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:28:42.558332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab712c86-093d-4934-aee5-ff990ce89958 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:42.558636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab712c86-093d-4934-aee5-ff990ce89958 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:42.558930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab712c86-093d-4934-aee5-ff990ce89958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:42.578997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-ab712c86-093d-4934-aee5-ff990ce89958 None None] MFA Rules not processed for user `327b347d6295422c99ce7b44eab00467`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:42.580208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-ab712c86-093d-4934-aee5-ff990ce89958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:42.586719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 737/1474] 10.222.0.40 () {58 vars in 1059 bytes} [Thu Jun 16 17:28:42 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:42.593955 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6f2d5-3989-490f-946b-e34759333a96 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:42.604332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6f2d5-3989-490f-946b-e34759333a96 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:42.607861 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca7cc9-18a3-415e-9c5d-5782ef3ad187 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327b347d6295422c99ce7b44eab00467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:42.608498 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca7cc9-18a3-415e-9c5d-5782ef3ad187 service glance] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:42.609358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca7cc9-18a3-415e-9c5d-5782ef3ad187 service glance] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:42.609645 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca7cc9-18a3-415e-9c5d-5782ef3ad187 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:42.618531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca7cc9-18a3-415e-9c5d-5782ef3ad187 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:42.619964 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca7cc9-18a3-415e-9c5d-5782ef3ad187 service glance] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:42.630233 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 738/1475] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:44.895000 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572278ba-b3d7-4bc0-b54f-d57c1b0bf145 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:44.895585 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572278ba-b3d7-4bc0-b54f-d57c1b0bf145 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:44.895585 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572278ba-b3d7-4bc0-b54f-d57c1b0bf145 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:44.897166 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 738/1476] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:28:44.902586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31de905-e9b3-44d1-9167-e7d81ba26ccf None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:44.903007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31de905-e9b3-44d1-9167-e7d81ba26ccf None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:44.903128 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31de905-e9b3-44d1-9167-e7d81ba26ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:44.920748 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e31de905-e9b3-44d1-9167-e7d81ba26ccf None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:44.923059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e31de905-e9b3-44d1-9167-e7d81ba26ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:44.930875 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 739/1477] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:28:44.955404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572278ba-b3d7-4bc0-b54f-d57c1b0bf145 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:44.967658 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572278ba-b3d7-4bc0-b54f-d57c1b0bf145 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:44.972169 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d328d-f9da-4594-a9d8-eeddc40caeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:44.972816 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d328d-f9da-4594-a9d8-eeddc40caeb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:44.973105 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d328d-f9da-4594-a9d8-eeddc40caeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:44.973418 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d328d-f9da-4594-a9d8-eeddc40caeb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:44.974156 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d328d-f9da-4594-a9d8-eeddc40caeb6 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:44.975953 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: WARNING py.warnings [None req-f07d328d-f9da-4594-a9d8-eeddc40caeb6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:28:44.975953 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: warnings.warn(msg) Jun 16 17:28:44.975953 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: Jun 16 17:28:44.977101 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d328d-f9da-4594-a9d8-eeddc40caeb6 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:44.996901 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 739/1478] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 17:28:44 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:28:47.167110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8589f-780d-4def-809e-c041b6363fe0 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:47.167678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8589f-780d-4def-809e-c041b6363fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:47.167678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8589f-780d-4def-809e-c041b6363fe0 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:47.169065 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 740/1479] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:28:47.174225 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdc1c3-d95c-4851-b284-b22286fdb2a2 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:47.174639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdc1c3-d95c-4851-b284-b22286fdb2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:47.174796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdc1c3-d95c-4851-b284-b22286fdb2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:47.192289 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-0cbdc1c3-d95c-4851-b284-b22286fdb2a2 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:47.193518 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0cbdc1c3-d95c-4851-b284-b22286fdb2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:47.204676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 740/1480] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:47.249000 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8589f-780d-4def-809e-c041b6363fe0 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:47.264352 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8589f-780d-4def-809e-c041b6363fe0 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:47.271220 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5a1c1-1b9d-4a76-a99d-27fe3f6029ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:47.271875 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5a1c1-1b9d-4a76-a99d-27fe3f6029ad service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:47.272168 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5a1c1-1b9d-4a76-a99d-27fe3f6029ad service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:47.272449 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5a1c1-1b9d-4a76-a99d-27fe3f6029ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:47.280799 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5a1c1-1b9d-4a76-a99d-27fe3f6029ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:47.282573 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5a1c1-1b9d-4a76-a99d-27fe3f6029ad service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:47.292172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 741/1481] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:49.619313 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ff185-6795-47b7-bca8-e8cc85e348aa None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:49.620093 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ff185-6795-47b7-bca8-e8cc85e348aa None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:49.620093 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ff185-6795-47b7-bca8-e8cc85e348aa None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:49.621078 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 741/1482] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:28:49.626511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbc55d-334d-47df-a620-6f05d97f7ca1 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:49.626966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbc55d-334d-47df-a620-6f05d97f7ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:49.627076 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbc55d-334d-47df-a620-6f05d97f7ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:49.644647 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-2ecbc55d-334d-47df-a620-6f05d97f7ca1 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:49.645878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2ecbc55d-334d-47df-a620-6f05d97f7ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:49.653555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 742/1483] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:49.689464 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ff185-6795-47b7-bca8-e8cc85e348aa None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:49.700260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ff185-6795-47b7-bca8-e8cc85e348aa None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:49.703827 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08260f92-391b-4af3-97cd-357039fdbc25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:49.704456 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08260f92-391b-4af3-97cd-357039fdbc25 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:49.704743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08260f92-391b-4af3-97cd-357039fdbc25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:49.705024 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08260f92-391b-4af3-97cd-357039fdbc25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:49.713991 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08260f92-391b-4af3-97cd-357039fdbc25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:49.715401 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08260f92-391b-4af3-97cd-357039fdbc25 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:49.725405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 742/1484] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:51.926076 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb6aa8-ca70-47fa-8cde-206db8d45483 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:51.926707 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb6aa8-ca70-47fa-8cde-206db8d45483 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:51.926707 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb6aa8-ca70-47fa-8cde-206db8d45483 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:51.928079 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 743/1485] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:28:51.934998 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49f97a-131e-496c-b025-b45f6716cdb9 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:51.935481 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49f97a-131e-496c-b025-b45f6716cdb9 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:51.935594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49f97a-131e-496c-b025-b45f6716cdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:51.954510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-0c49f97a-131e-496c-b025-b45f6716cdb9 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:51.955717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0c49f97a-131e-496c-b025-b45f6716cdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:51.963894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 743/1486] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:52.002147 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb6aa8-ca70-47fa-8cde-206db8d45483 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:52.012931 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb6aa8-ca70-47fa-8cde-206db8d45483 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:52.017123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf101979-3488-4a83-a2f4-b16e7e599584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:52.017812 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf101979-3488-4a83-a2f4-b16e7e599584 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:52.018093 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf101979-3488-4a83-a2f4-b16e7e599584 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:52.018433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf101979-3488-4a83-a2f4-b16e7e599584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:52.027951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf101979-3488-4a83-a2f4-b16e7e599584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:52.029424 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf101979-3488-4a83-a2f4-b16e7e599584 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:52.039498 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 744/1487] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:54.277051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348dbe3-4718-48a3-b08b-00d9ccd52b7b None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:54.277625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348dbe3-4718-48a3-b08b-00d9ccd52b7b None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:54.277625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348dbe3-4718-48a3-b08b-00d9ccd52b7b None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:54.279005 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 744/1488] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:28:54.284588 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85364e77-b0e4-4777-ae3f-928f36a7139f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:54.284990 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85364e77-b0e4-4777-ae3f-928f36a7139f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:54.285181 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85364e77-b0e4-4777-ae3f-928f36a7139f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:54.303435 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-85364e77-b0e4-4777-ae3f-928f36a7139f None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:54.304632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-85364e77-b0e4-4777-ae3f-928f36a7139f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:54.312458 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 745/1489] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:54.352110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348dbe3-4718-48a3-b08b-00d9ccd52b7b None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:54.364375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348dbe3-4718-48a3-b08b-00d9ccd52b7b None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:54.367953 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd04b8d-e319-4e0a-ba6d-89346a399334 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:54.368605 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd04b8d-e319-4e0a-ba6d-89346a399334 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:54.368902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd04b8d-e319-4e0a-ba6d-89346a399334 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:54.369218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd04b8d-e319-4e0a-ba6d-89346a399334 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:54.379054 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd04b8d-e319-4e0a-ba6d-89346a399334 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:54.380527 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd04b8d-e319-4e0a-ba6d-89346a399334 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:54.390101 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 745/1490] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:56.911738 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc4386-cee7-4780-867a-75f8add9f7e0 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:56.912426 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc4386-cee7-4780-867a-75f8add9f7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:56.912426 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc4386-cee7-4780-867a-75f8add9f7e0 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:56.913989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 746/1491] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 17:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:28:56.919109 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709dfe5-ec66-4bd6-88b1-481cfac73c59 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:56.919513 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709dfe5-ec66-4bd6-88b1-481cfac73c59 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:56.919747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709dfe5-ec66-4bd6-88b1-481cfac73c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:56.937422 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-8709dfe5-ec66-4bd6-88b1-481cfac73c59 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:56.938630 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-8709dfe5-ec66-4bd6-88b1-481cfac73c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:56.946432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 746/1492] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 17:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:56.985705 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc4386-cee7-4780-867a-75f8add9f7e0 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:56.996457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc4386-cee7-4780-867a-75f8add9f7e0 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:57.000979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a4b44-7efe-4361-ac5d-32bafc7b5775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:57.001680 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a4b44-7efe-4361-ac5d-32bafc7b5775 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:57.001969 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a4b44-7efe-4361-ac5d-32bafc7b5775 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:57.002251 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a4b44-7efe-4361-ac5d-32bafc7b5775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:57.013696 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a4b44-7efe-4361-ac5d-32bafc7b5775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:57.015188 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a4b44-7efe-4361-ac5d-32bafc7b5775 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:57.026134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 747/1493] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:29:06.932218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=33 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:07.021563 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=33 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:23.884807 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98f4a7-cfea-400b-aee8-a68bfc66ff69 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:23.885429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98f4a7-cfea-400b-aee8-a68bfc66ff69 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:23.885429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98f4a7-cfea-400b-aee8-a68bfc66ff69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:23.903935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-6a98f4a7-cfea-400b-aee8-a68bfc66ff69 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:23.905216 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-6a98f4a7-cfea-400b-aee8-a68bfc66ff69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:23.913059 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 747/1494] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:30:23.922162 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80725563-e8ce-4d33-8dca-c14b8d608cd5 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:23.922685 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80725563-e8ce-4d33-8dca-c14b8d608cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:23.922806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80725563-e8ce-4d33-8dca-c14b8d608cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:23.942045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-80725563-e8ce-4d33-8dca-c14b8d608cd5 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:23.943248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-80725563-e8ce-4d33-8dca-c14b8d608cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:23.951088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 748/1495] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:30:23.955652 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98f4a7-cfea-400b-aee8-a68bfc66ff69 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:23.965477 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98f4a7-cfea-400b-aee8-a68bfc66ff69 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:23.969972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49306ee-7876-4eee-8594-5aca3b8f2525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:23.970629 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49306ee-7876-4eee-8594-5aca3b8f2525 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:23.970921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49306ee-7876-4eee-8594-5aca3b8f2525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:23.971205 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49306ee-7876-4eee-8594-5aca3b8f2525 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:23.972570 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49306ee-7876-4eee-8594-5aca3b8f2525 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:23.972968 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49306ee-7876-4eee-8594-5aca3b8f2525 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:23.974037 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: WARNING py.warnings [None req-e49306ee-7876-4eee-8594-5aca3b8f2525 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:30:23.974037 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: warnings.warn(msg) Jun 16 17:30:23.974037 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: Jun 16 17:30:23.974919 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49306ee-7876-4eee-8594-5aca3b8f2525 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:23.986314 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 748/1496] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:23.995123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80725563-e8ce-4d33-8dca-c14b8d608cd5 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:24.004513 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80725563-e8ce-4d33-8dca-c14b8d608cd5 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:24.009169 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273fd405-36a6-4c42-b178-329afdb9bba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:24.009828 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fd405-36a6-4c42-b178-329afdb9bba3 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:24.010117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fd405-36a6-4c42-b178-329afdb9bba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:24.010397 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fd405-36a6-4c42-b178-329afdb9bba3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:24.011094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273fd405-36a6-4c42-b178-329afdb9bba3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:24.012888 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: WARNING py.warnings [None req-273fd405-36a6-4c42-b178-329afdb9bba3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:30:24.012888 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: warnings.warn(msg) Jun 16 17:30:24.012888 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: Jun 16 17:30:24.014200 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273fd405-36a6-4c42-b178-329afdb9bba3 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:24.050814 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 749/1497] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:30:23 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:30:24.055667 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49306ee-7876-4eee-8594-5aca3b8f2525 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:24.066073 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49306ee-7876-4eee-8594-5aca3b8f2525 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:24.123813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a4bcb-e812-4f16-bbcc-b4a2cef0e310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:24.124745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a4bcb-e812-4f16-bbcc-b4a2cef0e310 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:24.125033 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a4bcb-e812-4f16-bbcc-b4a2cef0e310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:24.125347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a4bcb-e812-4f16-bbcc-b4a2cef0e310 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:24.126038 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a4bcb-e812-4f16-bbcc-b4a2cef0e310 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:24.127047 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: WARNING py.warnings [None req-8c2a4bcb-e812-4f16-bbcc-b4a2cef0e310 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:30:24.127047 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: warnings.warn(msg) Jun 16 17:30:24.127047 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: Jun 16 17:30:24.128324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a4bcb-e812-4f16-bbcc-b4a2cef0e310 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:24.147749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 749/1498] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:30:24 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:30:24.151881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273fd405-36a6-4c42-b178-329afdb9bba3 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:24.162915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273fd405-36a6-4c42-b178-329afdb9bba3 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:24.167655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3b110-f2ab-45ae-bcf2-5bcee91a725d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:24.168326 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3b110-f2ab-45ae-bcf2-5bcee91a725d admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:24.168616 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3b110-f2ab-45ae-bcf2-5bcee91a725d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:24.168901 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3b110-f2ab-45ae-bcf2-5bcee91a725d admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:24.169667 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb3b110-f2ab-45ae-bcf2-5bcee91a725d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:24.170680 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: WARNING py.warnings [None req-7cb3b110-f2ab-45ae-bcf2-5bcee91a725d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:30:24.170680 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: warnings.warn(msg) Jun 16 17:30:24.170680 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: Jun 16 17:30:24.171562 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb3b110-f2ab-45ae-bcf2-5bcee91a725d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:24.184893 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 750/1499] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:30:24.189042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a4bcb-e812-4f16-bbcc-b4a2cef0e310 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:24.199545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a4bcb-e812-4f16-bbcc-b4a2cef0e310 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:24.204579 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb52800-545a-48d9-802b-b0ca09b49a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:24.205291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb52800-545a-48d9-802b-b0ca09b49a2b admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:24.205576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb52800-545a-48d9-802b-b0ca09b49a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:24.205863 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb52800-545a-48d9-802b-b0ca09b49a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:24.207092 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb52800-545a-48d9-802b-b0ca09b49a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:24.208114 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: WARNING py.warnings [None req-ddb52800-545a-48d9-802b-b0ca09b49a2b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:30:24.208114 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: warnings.warn(msg) Jun 16 17:30:24.208114 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: Jun 16 17:30:24.209004 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb52800-545a-48d9-802b-b0ca09b49a2b admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:24.223513 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 750/1500] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:30:24.228913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3b110-f2ab-45ae-bcf2-5bcee91a725d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:24.243405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3b110-f2ab-45ae-bcf2-5bcee91a725d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:24.251550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0995ea-0dd9-4101-9748-dce332e9db49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:24.253236 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0995ea-0dd9-4101-9748-dce332e9db49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:24.253846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0995ea-0dd9-4101-9748-dce332e9db49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:24.254372 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0995ea-0dd9-4101-9748-dce332e9db49 admin admin] PATH_INFO: `/v3/projects/95657c3b5b31481d9bed71cb968e8ef1/users/fb5b90689c374c628625ca31bc9d2b80/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:24.273280 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0995ea-0dd9-4101-9748-dce332e9db49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95657c3b5b31481d9bed71cb968e8ef1, user_id=fb5b90689c374c628625ca31bc9d2b80, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:24.274337 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: WARNING py.warnings [None req-ed0995ea-0dd9-4101-9748-dce332e9db49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:30:24.274337 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: warnings.warn(msg) Jun 16 17:30:24.274337 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: Jun 16 17:30:24.276279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0995ea-0dd9-4101-9748-dce332e9db49 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:24.292721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 751/1501] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:30:24 2022] PUT /identity/v3/projects/95657c3b5b31481d9bed71cb968e8ef1/users/fb5b90689c374c628625ca31bc9d2b80/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:30:24.304586 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442bcce-5036-454f-bf1f-d0bf08be1a33 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:24.304919 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442bcce-5036-454f-bf1f-d0bf08be1a33 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:24.305253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442bcce-5036-454f-bf1f-d0bf08be1a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:24.332711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-3442bcce-5036-454f-bf1f-d0bf08be1a33 None None] MFA Rules not processed for user `fb5b90689c374c628625ca31bc9d2b80`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:24.334013 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-3442bcce-5036-454f-bf1f-d0bf08be1a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:24.403381 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 751/1502] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:30:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:30:24.417058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96eed77-8a70-47af-8c03-d82b530958c2 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:24.417381 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96eed77-8a70-47af-8c03-d82b530958c2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:24.417662 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96eed77-8a70-47af-8c03-d82b530958c2 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:24.419045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 752/1503] 10.222.0.40 () {54 vars in 963 bytes} [Thu Jun 16 17:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:24.423918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccfa12-24e2-49fc-9e8e-89aee0708387 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:24.424249 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccfa12-24e2-49fc-9e8e-89aee0708387 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:24.424539 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccfa12-24e2-49fc-9e8e-89aee0708387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:24.443391 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-f0ccfa12-24e2-49fc-9e8e-89aee0708387 None None] MFA Rules not processed for user `1fadecafbe2549acaa4cde5412638d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:24.444635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-f0ccfa12-24e2-49fc-9e8e-89aee0708387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:24.489408 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:24.498284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 752/1504] 10.222.0.40 () {58 vars in 1060 bytes} [Thu Jun 16 17:30:24 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:30:24.506854 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96eed77-8a70-47af-8c03-d82b530958c2 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:24.517577 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96eed77-8a70-47af-8c03-d82b530958c2 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:24.521195 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b61b48-043d-4230-95c2-58ac8abbee59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:24.521846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b61b48-043d-4230-95c2-58ac8abbee59 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:24.522163 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b61b48-043d-4230-95c2-58ac8abbee59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:24.522468 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b61b48-043d-4230-95c2-58ac8abbee59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:24.530660 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b61b48-043d-4230-95c2-58ac8abbee59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:24.532071 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b61b48-043d-4230-95c2-58ac8abbee59 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:24.540774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 753/1505] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:30:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:30:24.583640 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dfbdf-90b2-4e1a-8fc9-1be0ebba32c2 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:24.583640 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dfbdf-90b2-4e1a-8fc9-1be0ebba32c2 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:24.583904 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dfbdf-90b2-4e1a-8fc9-1be0ebba32c2 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:24.585126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 753/1506] 10.222.0.40 () {52 vars in 829 bytes} [Thu Jun 16 17:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:24.588942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b61b48-043d-4230-95c2-58ac8abbee59 service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:24.650533 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b61b48-043d-4230-95c2-58ac8abbee59 service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:24.654949 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa905ae-05bc-4419-85dd-30b0570fa096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:24.655816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa905ae-05bc-4419-85dd-30b0570fa096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:24.656106 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa905ae-05bc-4419-85dd-30b0570fa096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:24.656387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa905ae-05bc-4419-85dd-30b0570fa096 admin admin] PATH_INFO: `/v3/users/fb5b90689c374c628625ca31bc9d2b80` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:24.657582 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa905ae-05bc-4419-85dd-30b0570fa096 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb5b90689c374c628625ca31bc9d2b80)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:24.658594 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: WARNING py.warnings [None req-aaa905ae-05bc-4419-85dd-30b0570fa096 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:30:24.658594 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: warnings.warn(msg) Jun 16 17:30:24.658594 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: Jun 16 17:30:24.660373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa905ae-05bc-4419-85dd-30b0570fa096 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:24.725047 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-aaa905ae-05bc-4419-85dd-30b0570fa096 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb5b90689c374c628625ca31bc9d2b80'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:24.743591 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-aaa905ae-05bc-4419-85dd-30b0570fa096 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb5b90689c374c628625ca31bc9d2b80'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:24.744273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-aaa905ae-05bc-4419-85dd-30b0570fa096 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb5b90689c374c628625ca31bc9d2b80'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:24.757515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-aaa905ae-05bc-4419-85dd-30b0570fa096 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb5b90689c374c628625ca31bc9d2b80'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:24.761419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:24.767846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-aaa905ae-05bc-4419-85dd-30b0570fa096 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb5b90689c374c628625ca31bc9d2b80'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:24.775580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 754/1507] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:30:24 2022] DELETE /identity/v3/users/fb5b90689c374c628625ca31bc9d2b80 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:24.779948 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed64f2-6ee4-42ae-9a92-ff6f3fe7d95e None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:24.780252 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed64f2-6ee4-42ae-9a92-ff6f3fe7d95e None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:24.780544 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed64f2-6ee4-42ae-9a92-ff6f3fe7d95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:24.800647 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-37ed64f2-6ee4-42ae-9a92-ff6f3fe7d95e None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:24.801931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-37ed64f2-6ee4-42ae-9a92-ff6f3fe7d95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:24.861851 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 754/1508] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:30:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:30:24.872636 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa905ae-05bc-4419-85dd-30b0570fa096 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:24.874059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-aaa905ae-05bc-4419-85dd-30b0570fa096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:24.934572 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa905ae-05bc-4419-85dd-30b0570fa096 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:24.938228 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a10a2c-7d20-4d51-8678-53aaf0c99c8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:24.938895 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a10a2c-7d20-4d51-8678-53aaf0c99c8d service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:24.939213 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a10a2c-7d20-4d51-8678-53aaf0c99c8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:24.939497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a10a2c-7d20-4d51-8678-53aaf0c99c8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:24.949044 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a10a2c-7d20-4d51-8678-53aaf0c99c8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:24.950521 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a10a2c-7d20-4d51-8678-53aaf0c99c8d service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:24.960820 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 755/1509] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:30:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:30:25.317911 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed64f2-6ee4-42ae-9a92-ff6f3fe7d95e None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:25.319492 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-37ed64f2-6ee4-42ae-9a92-ff6f3fe7d95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:25.336213 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed64f2-6ee4-42ae-9a92-ff6f3fe7d95e None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:25.341374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d6a74-32b7-473b-8423-a5bd972f112e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:25.341962 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d6a74-32b7-473b-8423-a5bd972f112e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:25.342286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d6a74-32b7-473b-8423-a5bd972f112e admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:25.342571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d6a74-32b7-473b-8423-a5bd972f112e admin admin] PATH_INFO: `/v3/projects/95657c3b5b31481d9bed71cb968e8ef1` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:25.343786 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90d6a74-32b7-473b-8423-a5bd972f112e admin admin] RBAC: Authorizing `identity:delete_project(project_id=95657c3b5b31481d9bed71cb968e8ef1)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:25.344833 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: WARNING py.warnings [None req-d90d6a74-32b7-473b-8423-a5bd972f112e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:30:25.344833 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: warnings.warn(msg) Jun 16 17:30:25.344833 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: Jun 16 17:30:25.346113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90d6a74-32b7-473b-8423-a5bd972f112e admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:25.413263 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d90d6a74-32b7-473b-8423-a5bd972f112e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95657c3b5b31481d9bed71cb968e8ef1', 'request_id': 'req-d90d6a74-32b7-473b-8423-a5bd972f112e'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:25.419295 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d90d6a74-32b7-473b-8423-a5bd972f112e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95657c3b5b31481d9bed71cb968e8ef1', 'request_id': 'req-d90d6a74-32b7-473b-8423-a5bd972f112e'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:25.433340 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d90d6a74-32b7-473b-8423-a5bd972f112e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95657c3b5b31481d9bed71cb968e8ef1', 'request_id': 'req-d90d6a74-32b7-473b-8423-a5bd972f112e'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:25.450856 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d90d6a74-32b7-473b-8423-a5bd972f112e admin admin] The token cache is being invalidate because project 95657c3b5b31481d9bed71cb968e8ef1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:30:25.451154 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d90d6a74-32b7-473b-8423-a5bd972f112e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:25.451884 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d90d6a74-32b7-473b-8423-a5bd972f112e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:25.453660 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 755/1510] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:30:25 2022] DELETE /identity/v3/projects/95657c3b5b31481d9bed71cb968e8ef1 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:30:34.955715 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:35.447668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:11.610916 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb94d0d-814c-45aa-8332-647c18f0a102 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:11.612089 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb94d0d-814c-45aa-8332-647c18f0a102 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:11.612217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb94d0d-814c-45aa-8332-647c18f0a102 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:11.613626 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:32:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:11.618391 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13724771-ed94-46e0-9e51-0ee87267d2a8 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:11.618829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13724771-ed94-46e0-9e51-0ee87267d2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:11.618944 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13724771-ed94-46e0-9e51-0ee87267d2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:11.636756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-13724771-ed94-46e0-9e51-0ee87267d2a8 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:11.637994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-13724771-ed94-46e0-9e51-0ee87267d2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:11.696636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:32:11.870925 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb94d0d-814c-45aa-8332-647c18f0a102 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:11.872742 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9eb94d0d-814c-45aa-8332-647c18f0a102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:11.952058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb94d0d-814c-45aa-8332-647c18f0a102 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:11.957642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245bfd7-4697-411f-ab1b-ed4fe7c56f70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:11.958304 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245bfd7-4697-411f-ab1b-ed4fe7c56f70 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:11.958594 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245bfd7-4697-411f-ab1b-ed4fe7c56f70 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:11.958879 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245bfd7-4697-411f-ab1b-ed4fe7c56f70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:11.966902 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245bfd7-4697-411f-ab1b-ed4fe7c56f70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:11.968527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245bfd7-4697-411f-ab1b-ed4fe7c56f70 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:11.978905 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 757/1513] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:32:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:32:12.857747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842c30e-fb08-4dd1-bead-75b987d32214 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:12.858281 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842c30e-fb08-4dd1-bead-75b987d32214 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:12.858403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842c30e-fb08-4dd1-bead-75b987d32214 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:12.859676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:12.864644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43dc6c-46cc-43ed-9527-efee457b7ca5 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:12.865099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43dc6c-46cc-43ed-9527-efee457b7ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:12.865331 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43dc6c-46cc-43ed-9527-efee457b7ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:12.885254 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-3f43dc6c-46cc-43ed-9527-efee457b7ca5 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:12.886509 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3f43dc6c-46cc-43ed-9527-efee457b7ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:12.895703 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:32:13.050474 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842c30e-fb08-4dd1-bead-75b987d32214 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:13.065306 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842c30e-fb08-4dd1-bead-75b987d32214 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:13.074031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8d653-9147-4b96-8389-b877805197af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:13.074693 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8d653-9147-4b96-8389-b877805197af service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:13.074970 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8d653-9147-4b96-8389-b877805197af service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:13.075252 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8d653-9147-4b96-8389-b877805197af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:13.084190 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8d653-9147-4b96-8389-b877805197af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:13.085831 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8d653-9147-4b96-8389-b877805197af service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:13.095249 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 758/1516] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:32:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:32:13.230801 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe9b2f-d532-4905-a494-404ec14b50cc None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:13.231077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe9b2f-d532-4905-a494-404ec14b50cc None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:13.231362 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe9b2f-d532-4905-a494-404ec14b50cc None None] PATH_INFO: `/v3` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:13.232745 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 759/1517] 10.222.0.47 () {54 vars in 925 bytes} [Thu Jun 16 17:32:13 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:13.239643 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28036e-c00e-4df8-9b92-094ec06141b1 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:13.239948 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28036e-c00e-4df8-9b92-094ec06141b1 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:13.240238 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28036e-c00e-4df8-9b92-094ec06141b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:13.259119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-1d28036e-c00e-4df8-9b92-094ec06141b1 None None] MFA Rules not processed for user `75f0586a8c1941448d750918183f6dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:13.260307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-1d28036e-c00e-4df8-9b92-094ec06141b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:13.322263 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 759/1518] 10.222.0.47 () {58 vars in 1015 bytes} [Thu Jun 16 17:32:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:32:17.051029 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad99ca-109c-41fb-82a0-a528afbb0046 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:17.051576 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad99ca-109c-41fb-82a0-a528afbb0046 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:17.051692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad99ca-109c-41fb-82a0-a528afbb0046 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:17.053057 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 760/1519] 10.222.0.40 () {54 vars in 960 bytes} [Thu Jun 16 17:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:17.058426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137cbe4-949a-47ec-aa6c-63983af65037 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:17.058924 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137cbe4-949a-47ec-aa6c-63983af65037 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:17.059033 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137cbe4-949a-47ec-aa6c-63983af65037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:17.078581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-0137cbe4-949a-47ec-aa6c-63983af65037 None None] MFA Rules not processed for user `75f0586a8c1941448d750918183f6dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:17.080530 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0137cbe4-949a-47ec-aa6c-63983af65037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:17.089682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 760/1520] 10.222.0.40 () {58 vars in 1057 bytes} [Thu Jun 16 17:32:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:32:17.098077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad99ca-109c-41fb-82a0-a528afbb0046 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:17.111636 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad99ca-109c-41fb-82a0-a528afbb0046 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:17.116542 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfebfb2-9556-4fbf-b249-063a53ef2c65 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75f0586a8c1941448d750918183f6dc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:17.117211 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfebfb2-9556-4fbf-b249-063a53ef2c65 service placement] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:17.117499 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfebfb2-9556-4fbf-b249-063a53ef2c65 service placement] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:17.117777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfebfb2-9556-4fbf-b249-063a53ef2c65 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:17.126715 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfebfb2-9556-4fbf-b249-063a53ef2c65 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:17.128307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfebfb2-9556-4fbf-b249-063a53ef2c65 service placement] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:17.138803 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 761/1521] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:32:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:32:24.389442 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e3524-cafd-48ca-97ad-bc2ab2b83f60 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:24.389968 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e3524-cafd-48ca-97ad-bc2ab2b83f60 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:24.390072 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e3524-cafd-48ca-97ad-bc2ab2b83f60 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:24.391385 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:24.396351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb2721-ffe3-4d00-bebc-f11788da2cd3 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:24.396819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb2721-ffe3-4d00-bebc-f11788da2cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:24.396932 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb2721-ffe3-4d00-bebc-f11788da2cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:24.415344 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-59fb2721-ffe3-4d00-bebc-f11788da2cd3 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:24.416546 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-59fb2721-ffe3-4d00-bebc-f11788da2cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:24.425275 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:32:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:32:24.580227 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e3524-cafd-48ca-97ad-bc2ab2b83f60 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:24.594996 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e3524-cafd-48ca-97ad-bc2ab2b83f60 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:24.601258 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2e9b5-a85f-46d4-92ad-68e8313742c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:24.601909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2e9b5-a85f-46d4-92ad-68e8313742c0 service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:24.602199 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2e9b5-a85f-46d4-92ad-68e8313742c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:24.602484 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2e9b5-a85f-46d4-92ad-68e8313742c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:24.610578 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2e9b5-a85f-46d4-92ad-68e8313742c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:24.612930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2e9b5-a85f-46d4-92ad-68e8313742c0 service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:24.622416 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 762/1524] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:32:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:32:25.494885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea5b1b-9ac1-452e-8acc-6b56eaa2e84d None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:25.495419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea5b1b-9ac1-452e-8acc-6b56eaa2e84d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:25.495419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea5b1b-9ac1-452e-8acc-6b56eaa2e84d None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:25.496786 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:25.501880 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52c93b-ef31-41d6-b5c2-6ce76faa34ce None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:25.502356 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52c93b-ef31-41d6-b5c2-6ce76faa34ce None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:25.502596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52c93b-ef31-41d6-b5c2-6ce76faa34ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:25.520722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-1d52c93b-ef31-41d6-b5c2-6ce76faa34ce None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:25.521955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-1d52c93b-ef31-41d6-b5c2-6ce76faa34ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:25.529782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:32:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:32:25.694227 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea5b1b-9ac1-452e-8acc-6b56eaa2e84d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:25.708607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea5b1b-9ac1-452e-8acc-6b56eaa2e84d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:25.714058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7df91-b59b-46b3-9038-7ff8c90c6ffa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:25.714707 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7df91-b59b-46b3-9038-7ff8c90c6ffa service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:25.714994 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7df91-b59b-46b3-9038-7ff8c90c6ffa service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:25.715276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7df91-b59b-46b3-9038-7ff8c90c6ffa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:25.723911 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d7df91-b59b-46b3-9038-7ff8c90c6ffa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:25.725539 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d7df91-b59b-46b3-9038-7ff8c90c6ffa service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:25.734878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 764/1527] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:32:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:32:29.732847 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f55dc7-1ae5-4c5f-826f-af4585d01363 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:29.733473 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f55dc7-1ae5-4c5f-826f-af4585d01363 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:29.733473 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f55dc7-1ae5-4c5f-826f-af4585d01363 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:29.735067 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:29.740236 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60b67d-950a-479c-8170-932606287ced None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:29.740685 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60b67d-950a-479c-8170-932606287ced None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:29.740790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60b67d-950a-479c-8170-932606287ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:29.759936 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-5e60b67d-950a-479c-8170-932606287ced None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:29.761163 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5e60b67d-950a-479c-8170-932606287ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:29.770075 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:32:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:32:29.808292 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f55dc7-1ae5-4c5f-826f-af4585d01363 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:29.810021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-a8f55dc7-1ae5-4c5f-826f-af4585d01363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:29.868673 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f55dc7-1ae5-4c5f-826f-af4585d01363 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:29.872170 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8adcdc-2c07-440f-8f5d-ff0e2c94ae5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:29.872814 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8adcdc-2c07-440f-8f5d-ff0e2c94ae5e service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:29.873103 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8adcdc-2c07-440f-8f5d-ff0e2c94ae5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:29.873423 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8adcdc-2c07-440f-8f5d-ff0e2c94ae5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:29.884135 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8adcdc-2c07-440f-8f5d-ff0e2c94ae5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:29.885978 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8adcdc-2c07-440f-8f5d-ff0e2c94ae5e service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:29.891171 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:29.897337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 765/1530] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:32:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:32:32.016328 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4a07d-88f7-4afe-b710-dd5a5d163bb1 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:32.016884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4a07d-88f7-4afe-b710-dd5a5d163bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:32.017008 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4a07d-88f7-4afe-b710-dd5a5d163bb1 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:32.018930 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:32:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:32.024144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f7d17-7520-4425-907b-1535239c0e58 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:32.024613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f7d17-7520-4425-907b-1535239c0e58 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:32.024720 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f7d17-7520-4425-907b-1535239c0e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:32.050301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-5e6f7d17-7520-4425-907b-1535239c0e58 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:32.051551 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-5e6f7d17-7520-4425-907b-1535239c0e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:32.062411 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:32:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:32:32.105312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4a07d-88f7-4afe-b710-dd5a5d163bb1 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:32.107061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-afd4a07d-88f7-4afe-b710-dd5a5d163bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:32.120342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4a07d-88f7-4afe-b710-dd5a5d163bb1 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:32.123922 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff599a0-5ab5-4444-a10e-4719ba29f460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:32.124820 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff599a0-5ab5-4444-a10e-4719ba29f460 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:32.125107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff599a0-5ab5-4444-a10e-4719ba29f460 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:32.125416 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff599a0-5ab5-4444-a10e-4719ba29f460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:32.135164 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff599a0-5ab5-4444-a10e-4719ba29f460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:32.136583 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff599a0-5ab5-4444-a10e-4719ba29f460 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:32.146557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 767/1533] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:32:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:32:35.806171 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cac46b-5383-4d21-94c6-22b649ee3c16 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:35.806824 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cac46b-5383-4d21-94c6-22b649ee3c16 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:35.806824 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cac46b-5383-4d21-94c6-22b649ee3c16 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:35.808112 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:35.813688 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330016c-8e1b-47ca-a0e6-040a8d9fecc9 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:35.814107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330016c-8e1b-47ca-a0e6-040a8d9fecc9 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:35.814264 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330016c-8e1b-47ca-a0e6-040a8d9fecc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:35.820648 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=62 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:35.833205 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d330016c-8e1b-47ca-a0e6-040a8d9fecc9 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:35.834423 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d330016c-8e1b-47ca-a0e6-040a8d9fecc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:35.842270 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:32:36.114779 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cac46b-5383-4d21-94c6-22b649ee3c16 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:36.116451 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-74cac46b-5383-4d21-94c6-22b649ee3c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:36.174612 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cac46b-5383-4d21-94c6-22b649ee3c16 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:36.178294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3d67e-83f4-4d39-b618-4455b97a10eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:36.178941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3d67e-83f4-4d39-b618-4455b97a10eb service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:36.179231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3d67e-83f4-4d39-b618-4455b97a10eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:36.179516 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3d67e-83f4-4d39-b618-4455b97a10eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:36.188750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3d67e-83f4-4d39-b618-4455b97a10eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:36.190309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3d67e-83f4-4d39-b618-4455b97a10eb service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:36.199873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 768/1536] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:32:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:32:38.486930 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd3687-f2c1-4817-9fc8-7f396bda6961 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:38.487523 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd3687-f2c1-4817-9fc8-7f396bda6961 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:38.487523 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd3687-f2c1-4817-9fc8-7f396bda6961 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:38.488904 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:38.494359 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b509a-f776-4be3-bbcc-463ffb164706 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:38.494788 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b509a-f776-4be3-bbcc-463ffb164706 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:38.494920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b509a-f776-4be3-bbcc-463ffb164706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:38.512746 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-a27b509a-f776-4be3-bbcc-463ffb164706 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:38.513959 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-a27b509a-f776-4be3-bbcc-463ffb164706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:38.522316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:32:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:32:38.788028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd3687-f2c1-4817-9fc8-7f396bda6961 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:38.789882 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-33bd3687-f2c1-4817-9fc8-7f396bda6961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:38.803879 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd3687-f2c1-4817-9fc8-7f396bda6961 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:38.807562 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5ba70-d688-4537-9b79-cfca5135598a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:38.808217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5ba70-d688-4537-9b79-cfca5135598a service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:38.808510 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5ba70-d688-4537-9b79-cfca5135598a service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:38.808795 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5ba70-d688-4537-9b79-cfca5135598a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:38.818028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd5ba70-d688-4537-9b79-cfca5135598a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:38.819460 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd5ba70-d688-4537-9b79-cfca5135598a service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:38.828842 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 770/1539] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:32:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:32:48.507718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=32 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:48.824660 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=8 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:07.712043 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd54505-1920-4e1d-a944-104a913f52ee None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:07.712689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd54505-1920-4e1d-a944-104a913f52ee None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:07.712689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd54505-1920-4e1d-a944-104a913f52ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:07.731222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339f3a0-1ebb-45f7-99fe-0303506923d4 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:07.732312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339f3a0-1ebb-45f7-99fe-0303506923d4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:07.732826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339f3a0-1ebb-45f7-99fe-0303506923d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:07.749178 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-6fd54505-1920-4e1d-a944-104a913f52ee None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:07.749178 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-6fd54505-1920-4e1d-a944-104a913f52ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:07.766882 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e339f3a0-1ebb-45f7-99fe-0303506923d4 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:07.768489 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 770/1540] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:07.768838 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e339f3a0-1ebb-45f7-99fe-0303506923d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:07.780635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08ab2d-afd0-429e-936e-116363230e93 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:07.781118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08ab2d-afd0-429e-936e-116363230e93 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:07.781440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08ab2d-afd0-429e-936e-116363230e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:07.783178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 771/1541] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:34:07.794941 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc9ae3-6196-4bdc-8f3c-fbd48bf4c659 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:07.795604 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc9ae3-6196-4bdc-8f3c-fbd48bf4c659 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:07.796238 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc9ae3-6196-4bdc-8f3c-fbd48bf4c659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:07.810065 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-0a08ab2d-afd0-429e-936e-116363230e93 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:07.812109 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0a08ab2d-afd0-429e-936e-116363230e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:07.825186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 771/1542] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 16 17:34:07.827274 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08ab2d-afd0-429e-936e-116363230e93 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:07.829011 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-99cc9ae3-6196-4bdc-8f3c-fbd48bf4c659 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:07.835136 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-99cc9ae3-6196-4bdc-8f3c-fbd48bf4c659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:07.844669 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08ab2d-afd0-429e-936e-116363230e93 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:07.846164 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 772/1543] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:07.852277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc9ae3-6196-4bdc-8f3c-fbd48bf4c659 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:07.854041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ecb333-ab70-4fb1-a090-367cf00816e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:07.854743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecb333-ab70-4fb1-a090-367cf00816e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:07.855038 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecb333-ab70-4fb1-a090-367cf00816e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:07.855324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecb333-ab70-4fb1-a090-367cf00816e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:07.856077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecb333-ab70-4fb1-a090-367cf00816e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:07.856397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecb333-ab70-4fb1-a090-367cf00816e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:07.858067 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecb333-ab70-4fb1-a090-367cf00816e2 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:07.872016 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc9ae3-6196-4bdc-8f3c-fbd48bf4c659 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:07.877572 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19875d9-d51e-4739-bae6-5da952890308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:07.879232 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19875d9-d51e-4739-bae6-5da952890308 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:07.880004 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19875d9-d51e-4739-bae6-5da952890308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:07.880537 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19875d9-d51e-4739-bae6-5da952890308 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:07.881494 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 772/1544] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:34:07.881877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19875d9-d51e-4739-bae6-5da952890308 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:07.882437 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19875d9-d51e-4739-bae6-5da952890308 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:07.883690 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: WARNING py.warnings [None req-f19875d9-d51e-4739-bae6-5da952890308 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:34:07.883690 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: warnings.warn(msg) Jun 16 17:34:07.883690 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: Jun 16 17:34:07.885013 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19875d9-d51e-4739-bae6-5da952890308 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:07.893027 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ecb333-ab70-4fb1-a090-367cf00816e2 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:07.909689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 773/1545] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:07.918696 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19875d9-d51e-4739-bae6-5da952890308 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:07.923625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ecb333-ab70-4fb1-a090-367cf00816e2 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:07.935392 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19875d9-d51e-4739-bae6-5da952890308 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:07.937688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a06509-32ad-4bfd-91f1-e24da43c1591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:07.938405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a06509-32ad-4bfd-91f1-e24da43c1591 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:07.938698 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a06509-32ad-4bfd-91f1-e24da43c1591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:07.938985 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a06509-32ad-4bfd-91f1-e24da43c1591 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:07.939716 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a06509-32ad-4bfd-91f1-e24da43c1591 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:07.945176 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: WARNING py.warnings [None req-53a06509-32ad-4bfd-91f1-e24da43c1591 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:34:07.945176 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: warnings.warn(msg) Jun 16 17:34:07.945176 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: Jun 16 17:34:07.945176 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a06509-32ad-4bfd-91f1-e24da43c1591 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:07.953881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779039be-3544-425a-a2a2-2e2e9566ea61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:07.954699 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779039be-3544-425a-a2a2-2e2e9566ea61 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:07.955066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779039be-3544-425a-a2a2-2e2e9566ea61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:07.955512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779039be-3544-425a-a2a2-2e2e9566ea61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:07.956397 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779039be-3544-425a-a2a2-2e2e9566ea61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:07.960461 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779039be-3544-425a-a2a2-2e2e9566ea61 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.020959 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 774/1546] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:07 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:34:08.022196 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0a195-e0f2-4bf7-b110-dcf43da53d31 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.022535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0a195-e0f2-4bf7-b110-dcf43da53d31 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.022917 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0a195-e0f2-4bf7-b110-dcf43da53d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.039609 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 773/1547] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:07 2022] POST /identity/v3/projects => generated 367 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:08.041091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db265e5d-48e4-47d2-aa39-77a5b36a613b None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.041421 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db265e5d-48e4-47d2-aa39-77a5b36a613b None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.041708 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db265e5d-48e4-47d2-aa39-77a5b36a613b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.061872 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-5fd0a195-e0f2-4bf7-b110-dcf43da53d31 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:08.063170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5fd0a195-e0f2-4bf7-b110-dcf43da53d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:08.090390 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-db265e5d-48e4-47d2-aa39-77a5b36a613b None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:08.092480 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-db265e5d-48e4-47d2-aa39-77a5b36a613b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:08.150363 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 775/1548] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:08.151409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0a195-e0f2-4bf7-b110-dcf43da53d31 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.165501 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0a195-e0f2-4bf7-b110-dcf43da53d31 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.170535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfd8ca-eb84-4dee-b82d-cd11e17b04ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.171226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfd8ca-eb84-4dee-b82d-cd11e17b04ed admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.171630 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfd8ca-eb84-4dee-b82d-cd11e17b04ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.171952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfd8ca-eb84-4dee-b82d-cd11e17b04ed admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.172616 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bfd8ca-eb84-4dee-b82d-cd11e17b04ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.173728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: WARNING py.warnings [None req-a5bfd8ca-eb84-4dee-b82d-cd11e17b04ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:34:08.173728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: warnings.warn(msg) Jun 16 17:34:08.173728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: Jun 16 17:34:08.174955 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bfd8ca-eb84-4dee-b82d-cd11e17b04ed admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.176311 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 774/1549] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:08.177677 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db265e5d-48e4-47d2-aa39-77a5b36a613b None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.187233 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db265e5d-48e4-47d2-aa39-77a5b36a613b None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.193444 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334964b8-0f52-41d9-8f5c-84577575b2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.194211 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334964b8-0f52-41d9-8f5c-84577575b2f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.194653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334964b8-0f52-41d9-8f5c-84577575b2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.194968 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334964b8-0f52-41d9-8f5c-84577575b2f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.195822 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334964b8-0f52-41d9-8f5c-84577575b2f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.198044 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334964b8-0f52-41d9-8f5c-84577575b2f1 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.204354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 776/1550] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:08 2022] POST /identity/v3/users => generated 286 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:08.205862 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489e72a-6f95-4d14-a855-9f7b14504679 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.206159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489e72a-6f95-4d14-a855-9f7b14504679 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.206441 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489e72a-6f95-4d14-a855-9f7b14504679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.227316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 775/1551] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:08 2022] POST /identity/v3/users => generated 284 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:08.227316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eee049-822b-470f-b0af-b400279d3246 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.227661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eee049-822b-470f-b0af-b400279d3246 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.227932 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eee049-822b-470f-b0af-b400279d3246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.234399 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d489e72a-6f95-4d14-a855-9f7b14504679 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:08.235658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d489e72a-6f95-4d14-a855-9f7b14504679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:08.244601 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 777/1552] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:08.248960 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848d623-a326-44b3-bb1d-2655e9e25ca8 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.248960 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848d623-a326-44b3-bb1d-2655e9e25ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.248960 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848d623-a326-44b3-bb1d-2655e9e25ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.250013 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-f3eee049-822b-470f-b0af-b400279d3246 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:08.251315 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-f3eee049-822b-470f-b0af-b400279d3246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:08.262217 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 776/1553] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:08.262411 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccb145-3763-4fdb-921f-64791c888349 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.262902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccb145-3763-4fdb-921f-64791c888349 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.262902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccb145-3763-4fdb-921f-64791c888349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.267607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a848d623-a326-44b3-bb1d-2655e9e25ca8 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:08.269720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a848d623-a326-44b3-bb1d-2655e9e25ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:08.279593 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 778/1554] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:08.281228 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2eea4-13ac-4519-ba33-c6d918d52926 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.281523 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2eea4-13ac-4519-ba33-c6d918d52926 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.281655 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-adccb145-3763-4fdb-921f-64791c888349 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:08.281827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2eea4-13ac-4519-ba33-c6d918d52926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.282844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-adccb145-3763-4fdb-921f-64791c888349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:08.291402 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 777/1555] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:08.292201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccb145-3763-4fdb-921f-64791c888349 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.301979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-72c2eea4-13ac-4519-ba33-c6d918d52926 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:08.302160 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccb145-3763-4fdb-921f-64791c888349 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.303171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-72c2eea4-13ac-4519-ba33-c6d918d52926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:08.306699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b0072-ead1-4856-904b-8852655a0bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.307354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b0072-ead1-4856-904b-8852655a0bed admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.307643 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b0072-ead1-4856-904b-8852655a0bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.307923 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b0072-ead1-4856-904b-8852655a0bed admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.308654 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b0072-ead1-4856-904b-8852655a0bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.310278 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b0072-ead1-4856-904b-8852655a0bed admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.311429 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 779/1556] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:08.312639 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f10337-9bc5-4cc8-b209-c4821a2a3fd6 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.312928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f10337-9bc5-4cc8-b209-c4821a2a3fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.313238 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f10337-9bc5-4cc8-b209-c4821a2a3fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.325078 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 778/1557] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:08.325078 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b0072-ead1-4856-904b-8852655a0bed admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.331733 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-57f10337-9bc5-4cc8-b209-c4821a2a3fd6 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:08.332541 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b0072-ead1-4856-904b-8852655a0bed admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.332923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-57f10337-9bc5-4cc8-b209-c4821a2a3fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:08.337015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354db110-5a72-486d-8267-f5915985ab68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.337704 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354db110-5a72-486d-8267-f5915985ab68 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.337992 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354db110-5a72-486d-8267-f5915985ab68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.338274 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354db110-5a72-486d-8267-f5915985ab68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.338998 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354db110-5a72-486d-8267-f5915985ab68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.340574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354db110-5a72-486d-8267-f5915985ab68 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.340702 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 780/1558] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:08.342001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1193a94-d5d4-4034-b0cd-c6f3739a7f17 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.342288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1193a94-d5d4-4034-b0cd-c6f3739a7f17 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.342583 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1193a94-d5d4-4034-b0cd-c6f3739a7f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.356436 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 779/1559] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:08.357750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709c99f-0223-421e-af37-a2fb33880ac1 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.358043 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709c99f-0223-421e-af37-a2fb33880ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.358325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709c99f-0223-421e-af37-a2fb33880ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.370587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a1193a94-d5d4-4034-b0cd-c6f3739a7f17 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:08.371822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a1193a94-d5d4-4034-b0cd-c6f3739a7f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:08.379653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-b709c99f-0223-421e-af37-a2fb33880ac1 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:08.384684 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 781/1560] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:08.384864 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b709c99f-0223-421e-af37-a2fb33880ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:08.384970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cc589-7d35-481e-be7b-6d35e98551a0 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.384970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cc589-7d35-481e-be7b-6d35e98551a0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.384970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cc589-7d35-481e-be7b-6d35e98551a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.392383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 780/1561] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:08.393208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709c99f-0223-421e-af37-a2fb33880ac1 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.403053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709c99f-0223-421e-af37-a2fb33880ac1 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.407565 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753a73b-5efe-4398-b4b7-188766bc0681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.408232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753a73b-5efe-4398-b4b7-188766bc0681 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.408519 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753a73b-5efe-4398-b4b7-188766bc0681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.408802 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753a73b-5efe-4398-b4b7-188766bc0681 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.408998 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-393cc589-7d35-481e-be7b-6d35e98551a0 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:08.409572 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f753a73b-5efe-4398-b4b7-188766bc0681 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:08.409931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f753a73b-5efe-4398-b4b7-188766bc0681 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.410251 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-393cc589-7d35-481e-be7b-6d35e98551a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:08.411804 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f753a73b-5efe-4398-b4b7-188766bc0681 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.418518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 782/1562] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:08.423289 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cc589-7d35-481e-be7b-6d35e98551a0 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.424420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 781/1563] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:08.425119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753a73b-5efe-4398-b4b7-188766bc0681 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.434099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cc589-7d35-481e-be7b-6d35e98551a0 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.436322 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753a73b-5efe-4398-b4b7-188766bc0681 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.438984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9b3d6-5f6b-42c8-937e-2a0fe6fd839e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.439659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9b3d6-5f6b-42c8-937e-2a0fe6fd839e admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.439951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9b3d6-5f6b-42c8-937e-2a0fe6fd839e admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.440233 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9b3d6-5f6b-42c8-937e-2a0fe6fd839e admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.440896 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d94af4-0d1c-4f4f-8606-53ae2c1e7345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.441022 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9b3d6-5f6b-42c8-937e-2a0fe6fd839e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:08.441929 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9b3d6-5f6b-42c8-937e-2a0fe6fd839e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.442578 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d94af4-0d1c-4f4f-8606-53ae2c1e7345 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.442578 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d94af4-0d1c-4f4f-8606-53ae2c1e7345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.442771 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d94af4-0d1c-4f4f-8606-53ae2c1e7345 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.443430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d94af4-0d1c-4f4f-8606-53ae2c1e7345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.443551 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9b3d6-5f6b-42c8-937e-2a0fe6fd839e admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.445014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d94af4-0d1c-4f4f-8606-53ae2c1e7345 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.455406 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 783/1564] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:08.456020 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 782/1565] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:08.456749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d94af4-0d1c-4f4f-8606-53ae2c1e7345 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.457801 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297fa8f-0d2b-4e31-8cd0-fb4d1d7e99d6 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.458434 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297fa8f-0d2b-4e31-8cd0-fb4d1d7e99d6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.458926 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297fa8f-0d2b-4e31-8cd0-fb4d1d7e99d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.468111 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d94af4-0d1c-4f4f-8606-53ae2c1e7345 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.473570 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83024afd-f54d-4c0a-8828-8934f483f49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.474277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83024afd-f54d-4c0a-8828-8934f483f49f admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.474578 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83024afd-f54d-4c0a-8828-8934f483f49f admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.474858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83024afd-f54d-4c0a-8828-8934f483f49f admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.475597 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83024afd-f54d-4c0a-8828-8934f483f49f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.477214 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83024afd-f54d-4c0a-8828-8934f483f49f admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.477790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-9297fa8f-0d2b-4e31-8cd0-fb4d1d7e99d6 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:08.479017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9297fa8f-0d2b-4e31-8cd0-fb4d1d7e99d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:08.487482 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 783/1566] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:08.487630 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 784/1567] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:08.488980 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297fa8f-0d2b-4e31-8cd0-fb4d1d7e99d6 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.489935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83024afd-f54d-4c0a-8828-8934f483f49f admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.499586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297fa8f-0d2b-4e31-8cd0-fb4d1d7e99d6 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.500581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83024afd-f54d-4c0a-8828-8934f483f49f admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.504258 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0c755-7944-4328-8811-211dcc1144cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.504922 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0c755-7944-4328-8811-211dcc1144cd admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.505235 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0c755-7944-4328-8811-211dcc1144cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.505519 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0c755-7944-4328-8811-211dcc1144cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.505804 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681da4e-5f1e-4708-ae7d-edb148d0a284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.506255 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0c755-7944-4328-8811-211dcc1144cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:08.506464 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681da4e-5f1e-4708-ae7d-edb148d0a284 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.506563 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0c755-7944-4328-8811-211dcc1144cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.506757 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681da4e-5f1e-4708-ae7d-edb148d0a284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.507042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681da4e-5f1e-4708-ae7d-edb148d0a284 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.507782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681da4e-5f1e-4708-ae7d-edb148d0a284 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:08.508094 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681da4e-5f1e-4708-ae7d-edb148d0a284 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.508199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0c755-7944-4328-8811-211dcc1144cd admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.509742 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681da4e-5f1e-4708-ae7d-edb148d0a284 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.519901 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 785/1568] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:08.520705 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0c755-7944-4328-8811-211dcc1144cd admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.522752 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 784/1569] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:08.523768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681da4e-5f1e-4708-ae7d-edb148d0a284 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.531718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0c755-7944-4328-8811-211dcc1144cd admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.533354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:08.537278 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d207eed-0dc8-4882-bd8d-fb87c6aba160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.537960 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d207eed-0dc8-4882-bd8d-fb87c6aba160 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.538155 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681da4e-5f1e-4708-ae7d-edb148d0a284 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.538320 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d207eed-0dc8-4882-bd8d-fb87c6aba160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.538568 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d207eed-0dc8-4882-bd8d-fb87c6aba160 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.539307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d207eed-0dc8-4882-bd8d-fb87c6aba160 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:08.539623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d207eed-0dc8-4882-bd8d-fb87c6aba160 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.541243 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d207eed-0dc8-4882-bd8d-fb87c6aba160 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.544163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03458ba-a553-4663-ad22-d29cc066ec6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.544859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03458ba-a553-4663-ad22-d29cc066ec6c admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.545364 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03458ba-a553-4663-ad22-d29cc066ec6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.545650 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03458ba-a553-4663-ad22-d29cc066ec6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.546352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03458ba-a553-4663-ad22-d29cc066ec6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.548266 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03458ba-a553-4663-ad22-d29cc066ec6c admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.555022 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 786/1570] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:08.555815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d207eed-0dc8-4882-bd8d-fb87c6aba160 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.565357 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d207eed-0dc8-4882-bd8d-fb87c6aba160 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.571330 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e8946-ab9c-472f-9718-d39282cd490f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.572222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e8946-ab9c-472f-9718-d39282cd490f admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.572508 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e8946-ab9c-472f-9718-d39282cd490f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.572790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e8946-ab9c-472f-9718-d39282cd490f admin admin] PATH_INFO: `/v3/projects/7dc9438583194d70b654dd8ce36417e7/users/09185fe6124c4663b2f974616a8120b8/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.576096 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:08.578946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 785/1571] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:08 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:08.580025 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03458ba-a553-4663-ad22-d29cc066ec6c admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.589770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e8946-ab9c-472f-9718-d39282cd490f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dc9438583194d70b654dd8ce36417e7, user_id=09185fe6124c4663b2f974616a8120b8, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.592499 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e8946-ab9c-472f-9718-d39282cd490f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.606575 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 787/1572] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:08 2022] PUT /identity/v3/projects/7dc9438583194d70b654dd8ce36417e7/users/09185fe6124c4663b2f974616a8120b8/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:08.607359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e8946-ab9c-472f-9718-d39282cd490f admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.655791 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03458ba-a553-4663-ad22-d29cc066ec6c admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.672600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e8946-ab9c-472f-9718-d39282cd490f admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.677641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a75238-51bd-4345-85a5-28d13e5409e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.678462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a75238-51bd-4345-85a5-28d13e5409e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.678756 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a75238-51bd-4345-85a5-28d13e5409e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.679043 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a75238-51bd-4345-85a5-28d13e5409e2 admin admin] PATH_INFO: `/v3/projects/565a11ea3a9f475089782a186af4991d/users/fa4dfe8c58e8489ebcf5c6641bd34c2a/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.694962 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a75238-51bd-4345-85a5-28d13e5409e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=565a11ea3a9f475089782a186af4991d, user_id=fa4dfe8c58e8489ebcf5c6641bd34c2a, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.697670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a75238-51bd-4345-85a5-28d13e5409e2 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.713648 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 788/1573] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:08 2022] PUT /identity/v3/projects/565a11ea3a9f475089782a186af4991d/users/fa4dfe8c58e8489ebcf5c6641bd34c2a/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:08.714247 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a75238-51bd-4345-85a5-28d13e5409e2 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.730781 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d95d7a-f613-4478-8485-b122805a583d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.731461 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d95d7a-f613-4478-8485-b122805a583d admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.731755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d95d7a-f613-4478-8485-b122805a583d admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.732040 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d95d7a-f613-4478-8485-b122805a583d admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.732752 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d95d7a-f613-4478-8485-b122805a583d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.734793 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d95d7a-f613-4478-8485-b122805a583d admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.762689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 786/1574] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:08 2022] POST /identity/v3/projects => generated 375 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:08.763836 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d95d7a-f613-4478-8485-b122805a583d admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.790146 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a75238-51bd-4345-85a5-28d13e5409e2 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.827257 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d95d7a-f613-4478-8485-b122805a583d admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.832471 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52be41-fd71-42be-900b-dbd4a432480c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.833290 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52be41-fd71-42be-900b-dbd4a432480c admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.833745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52be41-fd71-42be-900b-dbd4a432480c admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.833954 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52be41-fd71-42be-900b-dbd4a432480c admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.834619 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52be41-fd71-42be-900b-dbd4a432480c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.836594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52be41-fd71-42be-900b-dbd4a432480c admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.849176 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70c196-b5ce-4e32-883c-49411addf997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.850025 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70c196-b5ce-4e32-883c-49411addf997 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.850263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70c196-b5ce-4e32-883c-49411addf997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.850545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70c196-b5ce-4e32-883c-49411addf997 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.851286 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70c196-b5ce-4e32-883c-49411addf997 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:08.851602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70c196-b5ce-4e32-883c-49411addf997 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.853280 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70c196-b5ce-4e32-883c-49411addf997 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.857361 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:08.864436 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 787/1575] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:08 2022] POST /identity/v3/projects => generated 393 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:08.865562 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52be41-fd71-42be-900b-dbd4a432480c admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.873609 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 789/1576] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:08.873609 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70c196-b5ce-4e32-883c-49411addf997 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.877620 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70c196-b5ce-4e32-883c-49411addf997 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.883224 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:08.937821 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52be41-fd71-42be-900b-dbd4a432480c admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.938409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b6ba8-8c78-4a12-be55-fd34bfec84f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.939088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b6ba8-8c78-4a12-be55-fd34bfec84f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.939383 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b6ba8-8c78-4a12-be55-fd34bfec84f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.939662 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b6ba8-8c78-4a12-be55-fd34bfec84f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.940362 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b6ba8-8c78-4a12-be55-fd34bfec84f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.942826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b6ba8-8c78-4a12-be55-fd34bfec84f4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.943201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace70832-7e88-4472-9a49-f6e4c37c88bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.943869 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace70832-7e88-4472-9a49-f6e4c37c88bd admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.944156 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace70832-7e88-4472-9a49-f6e4c37c88bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.944436 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace70832-7e88-4472-9a49-f6e4c37c88bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.945130 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace70832-7e88-4472-9a49-f6e4c37c88bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.947052 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace70832-7e88-4472-9a49-f6e4c37c88bd admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.971593 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 790/1577] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:08 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:08.972972 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b6ba8-8c78-4a12-be55-fd34bfec84f4 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.974619 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 788/1578] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:08 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:08.976105 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c468a-49eb-4fb3-aae2-73dd685d9b2f None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.977014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c468a-49eb-4fb3-aae2-73dd685d9b2f None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.977423 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c468a-49eb-4fb3-aae2-73dd685d9b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.004108 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-4c7c468a-49eb-4fb3-aae2-73dd685d9b2f None None] MFA Rules not processed for user `09185fe6124c4663b2f974616a8120b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:09.005389 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4c7c468a-49eb-4fb3-aae2-73dd685d9b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:09.040589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b6ba8-8c78-4a12-be55-fd34bfec84f4 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.072233 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 789/1579] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:09.076787 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e94bf-eb3f-491b-9f5e-b9b47c5e1c0d None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.076787 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e94bf-eb3f-491b-9f5e-b9b47c5e1c0d None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.076787 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e94bf-eb3f-491b-9f5e-b9b47c5e1c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.102822 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-bd4e94bf-eb3f-491b-9f5e-b9b47c5e1c0d None None] MFA Rules not processed for user `fa4dfe8c58e8489ebcf5c6641bd34c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:09.103448 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b61d1-036c-4d63-a357-6299ae63c031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.104074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-bd4e94bf-eb3f-491b-9f5e-b9b47c5e1c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:09.104187 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b61d1-036c-4d63-a357-6299ae63c031 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.104429 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b61d1-036c-4d63-a357-6299ae63c031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.104817 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b61d1-036c-4d63-a357-6299ae63c031 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.105459 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b61d1-036c-4d63-a357-6299ae63c031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.107400 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b61d1-036c-4d63-a357-6299ae63c031 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.131963 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 791/1580] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:08 2022] POST /identity/v3/users => generated 290 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:09.132785 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b61d1-036c-4d63-a357-6299ae63c031 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.143711 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b61d1-036c-4d63-a357-6299ae63c031 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.149636 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0870e8-4b66-4d48-93e9-1fe0ed4276c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.150345 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0870e8-4b66-4d48-93e9-1fe0ed4276c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.150635 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0870e8-4b66-4d48-93e9-1fe0ed4276c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.150923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0870e8-4b66-4d48-93e9-1fe0ed4276c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.151617 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0870e8-4b66-4d48-93e9-1fe0ed4276c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.153568 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0870e8-4b66-4d48-93e9-1fe0ed4276c6 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.174743 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 792/1581] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:09 2022] POST /identity/v3/users => generated 288 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:09.179596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 790/1582] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:09 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:09.179755 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0870e8-4b66-4d48-93e9-1fe0ed4276c6 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.179874 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e94bf-eb3f-491b-9f5e-b9b47c5e1c0d None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.187421 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:09.189514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0870e8-4b66-4d48-93e9-1fe0ed4276c6 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.192636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e94bf-eb3f-491b-9f5e-b9b47c5e1c0d None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.195580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3b25c-197b-4cdf-89e3-84a30be58a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.196261 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3b25c-197b-4cdf-89e3-84a30be58a3a admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.196554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3b25c-197b-4cdf-89e3-84a30be58a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.196836 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3b25c-197b-4cdf-89e3-84a30be58a3a admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.197632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3b25c-197b-4cdf-89e3-84a30be58a3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.198851 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81b7fb-0de6-444c-9a3c-b02dc4c93717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.199520 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81b7fb-0de6-444c-9a3c-b02dc4c93717 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.199632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3b25c-197b-4cdf-89e3-84a30be58a3a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.199812 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81b7fb-0de6-444c-9a3c-b02dc4c93717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.200095 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81b7fb-0de6-444c-9a3c-b02dc4c93717 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.200858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81b7fb-0de6-444c-9a3c-b02dc4c93717 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.202841 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81b7fb-0de6-444c-9a3c-b02dc4c93717 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.210396 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:09.221927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 793/1583] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:09 2022] POST /identity/v3/users => generated 297 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:09.223302 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3b25c-197b-4cdf-89e3-84a30be58a3a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.232013 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 791/1584] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:09 2022] POST /identity/v3/projects => generated 405 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:09.232490 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81b7fb-0de6-444c-9a3c-b02dc4c93717 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.239485 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3b25c-197b-4cdf-89e3-84a30be58a3a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.298942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46152c9-110f-4c7c-a6d6-931c7721aa59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.299534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81b7fb-0de6-444c-9a3c-b02dc4c93717 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.299673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46152c9-110f-4c7c-a6d6-931c7721aa59 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.299850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46152c9-110f-4c7c-a6d6-931c7721aa59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.300177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46152c9-110f-4c7c-a6d6-931c7721aa59 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.302059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46152c9-110f-4c7c-a6d6-931c7721aa59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.304006 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46152c9-110f-4c7c-a6d6-931c7721aa59 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.304901 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5670ca77-0e64-4c1a-823c-a18921f77dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.305713 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670ca77-0e64-4c1a-823c-a18921f77dc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.306036 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670ca77-0e64-4c1a-823c-a18921f77dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.306318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670ca77-0e64-4c1a-823c-a18921f77dc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.307013 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5670ca77-0e64-4c1a-823c-a18921f77dc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.308958 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5670ca77-0e64-4c1a-823c-a18921f77dc5 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.325207 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 794/1585] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:09 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:09.326472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa122a6-0dec-4856-a6d8-61b7a30c2de4 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.326771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa122a6-0dec-4856-a6d8-61b7a30c2de4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.327058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa122a6-0dec-4856-a6d8-61b7a30c2de4 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.328329 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 795/1586] 10.222.0.40 () {54 vars in 961 bytes} [Thu Jun 16 17:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:34:09.328674 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 792/1587] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:09 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:09.329032 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa122a6-0dec-4856-a6d8-61b7a30c2de4 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.330152 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5670ca77-0e64-4c1a-823c-a18921f77dc5 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.341056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5670ca77-0e64-4c1a-823c-a18921f77dc5 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.341905 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa122a6-0dec-4856-a6d8-61b7a30c2de4 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.345672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58384fe1-809a-4618-901d-61c7b395decb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.346457 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58384fe1-809a-4618-901d-61c7b395decb admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.346588 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d967d-d148-4cce-9c26-d18c86f92c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.346742 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58384fe1-809a-4618-901d-61c7b395decb admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.347024 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58384fe1-809a-4618-901d-61c7b395decb admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.347767 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d967d-d148-4cce-9c26-d18c86f92c60 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.348060 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d967d-d148-4cce-9c26-d18c86f92c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.348341 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d967d-d148-4cce-9c26-d18c86f92c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.348597 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58384fe1-809a-4618-901d-61c7b395decb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.349082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d967d-d148-4cce-9c26-d18c86f92c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.350350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58384fe1-809a-4618-901d-61c7b395decb admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.350748 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d967d-d148-4cce-9c26-d18c86f92c60 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.360997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 796/1588] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:09.361937 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 793/1589] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:09.362357 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d967d-d148-4cce-9c26-d18c86f92c60 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.362763 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58384fe1-809a-4618-901d-61c7b395decb admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.373745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58384fe1-809a-4618-901d-61c7b395decb admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.378459 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089712f-29e0-4cb6-9b71-5788b579f747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.379325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089712f-29e0-4cb6-9b71-5788b579f747 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.379727 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089712f-29e0-4cb6-9b71-5788b579f747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.380022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089712f-29e0-4cb6-9b71-5788b579f747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.380764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089712f-29e0-4cb6-9b71-5788b579f747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.382405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089712f-29e0-4cb6-9b71-5788b579f747 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.392874 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 794/1590] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:09.393675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089712f-29e0-4cb6-9b71-5788b579f747 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.404279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089712f-29e0-4cb6-9b71-5788b579f747 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.410598 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55272ad2-bead-435f-afc8-6fa455074807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.411503 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55272ad2-bead-435f-afc8-6fa455074807 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.411885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55272ad2-bead-435f-afc8-6fa455074807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.412251 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55272ad2-bead-435f-afc8-6fa455074807 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.412881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55272ad2-bead-435f-afc8-6fa455074807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.414856 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55272ad2-bead-435f-afc8-6fa455074807 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.422608 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d967d-d148-4cce-9c26-d18c86f92c60 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.427551 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9f8cb-a1cd-4365-8505-73a3c349cc4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.428240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9f8cb-a1cd-4365-8505-73a3c349cc4f service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.428530 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9f8cb-a1cd-4365-8505-73a3c349cc4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.428814 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9f8cb-a1cd-4365-8505-73a3c349cc4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.442369 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 795/1591] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:09 2022] POST /identity/v3/users => generated 303 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:09.442369 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55272ad2-bead-435f-afc8-6fa455074807 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.450583 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55272ad2-bead-435f-afc8-6fa455074807 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.457135 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ee313-ebfe-4c80-9112-aef7c38e23cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.458065 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ee313-ebfe-4c80-9112-aef7c38e23cd admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.458391 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ee313-ebfe-4c80-9112-aef7c38e23cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.458684 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ee313-ebfe-4c80-9112-aef7c38e23cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.459428 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ee313-ebfe-4c80-9112-aef7c38e23cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.461043 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ee313-ebfe-4c80-9112-aef7c38e23cd admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.471944 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 796/1592] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:09.473699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ee313-ebfe-4c80-9112-aef7c38e23cd admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.483289 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ee313-ebfe-4c80-9112-aef7c38e23cd admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.487777 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04ff01-5d35-4c48-9828-2ffec307d312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.489422 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04ff01-5d35-4c48-9828-2ffec307d312 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.490013 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04ff01-5d35-4c48-9828-2ffec307d312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.490306 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04ff01-5d35-4c48-9828-2ffec307d312 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.491036 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c04ff01-5d35-4c48-9828-2ffec307d312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.491162 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9f8cb-a1cd-4365-8505-73a3c349cc4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.492519 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9f8cb-a1cd-4365-8505-73a3c349cc4f service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.492895 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c04ff01-5d35-4c48-9828-2ffec307d312 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.501719 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 797/1593] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:09 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:09.502963 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5f418-0ee4-4556-b9c0-d84b4f0dd309 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.503256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5f418-0ee4-4556-b9c0-d84b4f0dd309 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.507056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 797/1594] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:09.507336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5f418-0ee4-4556-b9c0-d84b4f0dd309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.507435 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04ff01-5d35-4c48-9828-2ffec307d312 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.516581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04ff01-5d35-4c48-9828-2ffec307d312 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.522164 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7f7fd-3aa5-4cf6-b47c-984c8a0724df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.522840 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7f7fd-3aa5-4cf6-b47c-984c8a0724df admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.523131 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7f7fd-3aa5-4cf6-b47c-984c8a0724df admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.523416 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7f7fd-3aa5-4cf6-b47c-984c8a0724df admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.524152 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7f7fd-3aa5-4cf6-b47c-984c8a0724df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.524583 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-30d5f418-0ee4-4556-b9c0-d84b4f0dd309 None None] MFA Rules not processed for user `689a31e26c6b454a91c3fd79678b4d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:09.525830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-30d5f418-0ee4-4556-b9c0-d84b4f0dd309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:09.525954 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7f7fd-3aa5-4cf6-b47c-984c8a0724df admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.538031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 798/1595] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:09.539287 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7f7fd-3aa5-4cf6-b47c-984c8a0724df admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.553416 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7f7fd-3aa5-4cf6-b47c-984c8a0724df admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.555134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:09.559762 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62507d5-52a7-4fd9-bb77-aaee10ff9bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.560470 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62507d5-52a7-4fd9-bb77-aaee10ff9bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.560759 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62507d5-52a7-4fd9-bb77-aaee10ff9bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.561235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62507d5-52a7-4fd9-bb77-aaee10ff9bd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.561975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62507d5-52a7-4fd9-bb77-aaee10ff9bd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.564299 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62507d5-52a7-4fd9-bb77-aaee10ff9bd5 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.577944 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 799/1596] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:09.578597 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62507d5-52a7-4fd9-bb77-aaee10ff9bd5 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.589756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62507d5-52a7-4fd9-bb77-aaee10ff9bd5 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.594802 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31657125-9aa0-4a04-a79d-a3e88149858c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.595456 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31657125-9aa0-4a04-a79d-a3e88149858c admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.595750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31657125-9aa0-4a04-a79d-a3e88149858c admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.596034 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31657125-9aa0-4a04-a79d-a3e88149858c admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.596882 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31657125-9aa0-4a04-a79d-a3e88149858c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.599509 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31657125-9aa0-4a04-a79d-a3e88149858c admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.608645 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 798/1597] 10.222.0.40 () {58 vars in 1058 bytes} [Thu Jun 16 17:34:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:09.610144 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5f418-0ee4-4556-b9c0-d84b4f0dd309 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.613340 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 800/1598] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:09.614458 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31657125-9aa0-4a04-a79d-a3e88149858c admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.624499 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5f418-0ee4-4556-b9c0-d84b4f0dd309 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.625182 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31657125-9aa0-4a04-a79d-a3e88149858c admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.629694 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db48a95-1e39-4162-a705-72a35ec9eafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.630129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427f56b-9531-4e61-b436-1b49f6b16f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.630582 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db48a95-1e39-4162-a705-72a35ec9eafc admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.630915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db48a95-1e39-4162-a705-72a35ec9eafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.631025 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427f56b-9531-4e61-b436-1b49f6b16f89 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.631197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427f56b-9531-4e61-b436-1b49f6b16f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.631300 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db48a95-1e39-4162-a705-72a35ec9eafc admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.631486 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427f56b-9531-4e61-b436-1b49f6b16f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.632230 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427f56b-9531-4e61-b436-1b49f6b16f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.632809 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db48a95-1e39-4162-a705-72a35ec9eafc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.633889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427f56b-9531-4e61-b436-1b49f6b16f89 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.634462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db48a95-1e39-4162-a705-72a35ec9eafc admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.647867 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 801/1599] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:09.647867 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427f56b-9531-4e61-b436-1b49f6b16f89 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.649423 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 799/1600] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:09.650203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db48a95-1e39-4162-a705-72a35ec9eafc admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.658135 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427f56b-9531-4e61-b436-1b49f6b16f89 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.660728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db48a95-1e39-4162-a705-72a35ec9eafc admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.663961 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87bb55-97bb-4561-8e3f-f71312997e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.664710 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87bb55-97bb-4561-8e3f-f71312997e5e admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.665022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87bb55-97bb-4561-8e3f-f71312997e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.665328 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87bb55-97bb-4561-8e3f-f71312997e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.665653 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ff2e7-7cb9-4324-81df-48d0bc30ce5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.666070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87bb55-97bb-4561-8e3f-f71312997e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.666829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ff2e7-7cb9-4324-81df-48d0bc30ce5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.667190 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ff2e7-7cb9-4324-81df-48d0bc30ce5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.667546 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ff2e7-7cb9-4324-81df-48d0bc30ce5a admin admin] PATH_INFO: `/v3/projects/4537634610c645db8bdb7c704c09ee71/users/cbe7720ab8b7414a8c336de368ae1ec4/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.667689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87bb55-97bb-4561-8e3f-f71312997e5e admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.680019 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 802/1601] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:09.680019 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87bb55-97bb-4561-8e3f-f71312997e5e admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.688876 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ff2e7-7cb9-4324-81df-48d0bc30ce5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4537634610c645db8bdb7c704c09ee71, user_id=cbe7720ab8b7414a8c336de368ae1ec4, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.689819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87bb55-97bb-4561-8e3f-f71312997e5e admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.692573 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ff2e7-7cb9-4324-81df-48d0bc30ce5a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.695000 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829cd45-cbd5-4cbd-86cc-2995e7b9c346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.695899 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829cd45-cbd5-4cbd-86cc-2995e7b9c346 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.696192 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829cd45-cbd5-4cbd-86cc-2995e7b9c346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.696476 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829cd45-cbd5-4cbd-86cc-2995e7b9c346 admin admin] PATH_INFO: `/v3/projects/4904f0e2f64e4bef989f441380e843b5/users/3faab41439184fecbb3e1021eb81d555/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.710464 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:09.714793 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 800/1602] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:09 2022] PUT /identity/v3/projects/4537634610c645db8bdb7c704c09ee71/users/cbe7720ab8b7414a8c336de368ae1ec4/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:09.714793 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ff2e7-7cb9-4324-81df-48d0bc30ce5a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.716568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829cd45-cbd5-4cbd-86cc-2995e7b9c346 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4904f0e2f64e4bef989f441380e843b5, user_id=3faab41439184fecbb3e1021eb81d555, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.718089 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: WARNING py.warnings [None req-c829cd45-cbd5-4cbd-86cc-2995e7b9c346 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:34:09.718089 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: warnings.warn(msg) Jun 16 17:34:09.718089 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: Jun 16 17:34:09.720458 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829cd45-cbd5-4cbd-86cc-2995e7b9c346 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.737413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 803/1603] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:09 2022] PUT /identity/v3/projects/4904f0e2f64e4bef989f441380e843b5/users/3faab41439184fecbb3e1021eb81d555/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:09.744649 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829cd45-cbd5-4cbd-86cc-2995e7b9c346 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.792734 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ff2e7-7cb9-4324-81df-48d0bc30ce5a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.833570 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829cd45-cbd5-4cbd-86cc-2995e7b9c346 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.840977 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517ae8c-0460-40da-89da-62a3bc5c33ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.841684 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517ae8c-0460-40da-89da-62a3bc5c33ba service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.841973 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517ae8c-0460-40da-89da-62a3bc5c33ba service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.842261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517ae8c-0460-40da-89da-62a3bc5c33ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.866360 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab9042-a20b-45db-9e20-a07eb3c00ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:09.867766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab9042-a20b-45db-9e20-a07eb3c00ae6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:09.868059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab9042-a20b-45db-9e20-a07eb3c00ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:09.868341 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab9042-a20b-45db-9e20-a07eb3c00ae6 admin admin] PATH_INFO: `/v3/projects/d1c1d13142434356ac1210f14dad847b/users/a5c37b79bf3d40f4a6182cc296585c18/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:09.885748 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ab9042-a20b-45db-9e20-a07eb3c00ae6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1c1d13142434356ac1210f14dad847b, user_id=a5c37b79bf3d40f4a6182cc296585c18, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.888413 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ab9042-a20b-45db-9e20-a07eb3c00ae6 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.904531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 801/1604] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:09 2022] PUT /identity/v3/projects/d1c1d13142434356ac1210f14dad847b/users/a5c37b79bf3d40f4a6182cc296585c18/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:09.905392 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab9042-a20b-45db-9e20-a07eb3c00ae6 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.916231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517ae8c-0460-40da-89da-62a3bc5c33ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:09.917927 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517ae8c-0460-40da-89da-62a3bc5c33ba service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:09.931883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:09.934193 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 804/1605] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:09.943297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517ae8c-0460-40da-89da-62a3bc5c33ba service nova] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:09.957963 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517ae8c-0460-40da-89da-62a3bc5c33ba service nova] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:09.995773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab9042-a20b-45db-9e20-a07eb3c00ae6 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:10.002891 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c99d5-9899-4bf8-832a-37e482bc119a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:10.004121 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c99d5-9899-4bf8-832a-37e482bc119a admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:10.004727 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c99d5-9899-4bf8-832a-37e482bc119a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:10.005012 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c99d5-9899-4bf8-832a-37e482bc119a admin admin] PATH_INFO: `/v3/projects/d63c592c0afb4dfc9dd5b36be791decd/users/209ada6ba12b4bfda10045b4ee9301f1/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:10.028784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c99d5-9899-4bf8-832a-37e482bc119a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d63c592c0afb4dfc9dd5b36be791decd, user_id=209ada6ba12b4bfda10045b4ee9301f1, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:10.032199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c99d5-9899-4bf8-832a-37e482bc119a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:10.033968 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8266af-4909-4292-89a3-791049468f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:10.035302 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8266af-4909-4292-89a3-791049468f36 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:10.035700 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8266af-4909-4292-89a3-791049468f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:10.036430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8266af-4909-4292-89a3-791049468f36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:10.036930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8266af-4909-4292-89a3-791049468f36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:10.039026 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8266af-4909-4292-89a3-791049468f36 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:10.053896 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 802/1606] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:09 2022] PUT /identity/v3/projects/d63c592c0afb4dfc9dd5b36be791decd/users/209ada6ba12b4bfda10045b4ee9301f1/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:10.055037 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c99d5-9899-4bf8-832a-37e482bc119a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:10.065752 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 805/1607] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:10.067379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03e63e-e0fd-41b9-b8b8-a2731d9bb7e9 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:10.067994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03e63e-e0fd-41b9-b8b8-a2731d9bb7e9 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:10.068276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03e63e-e0fd-41b9-b8b8-a2731d9bb7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:10.092797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-4d03e63e-e0fd-41b9-b8b8-a2731d9bb7e9 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:10.094746 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4d03e63e-e0fd-41b9-b8b8-a2731d9bb7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:10.135734 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:10.170025 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c99d5-9899-4bf8-832a-37e482bc119a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:10.176462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe1378-4e3d-42e3-9219-dd8bea9dc57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:10.177595 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe1378-4e3d-42e3-9219-dd8bea9dc57d admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:10.178041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe1378-4e3d-42e3-9219-dd8bea9dc57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:10.178491 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe1378-4e3d-42e3-9219-dd8bea9dc57d admin admin] PATH_INFO: `/v3/projects/2a4ddecea1cd4714a724ffe5c1231e04/users/0bd4a401c3ae43f5b6fe0a5468af3ebb/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:10.199164 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe1378-4e3d-42e3-9219-dd8bea9dc57d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a4ddecea1cd4714a724ffe5c1231e04, user_id=0bd4a401c3ae43f5b6fe0a5468af3ebb, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:10.203421 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe1378-4e3d-42e3-9219-dd8bea9dc57d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:10.226937 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 806/1608] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:10.228697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44167ce8-6728-4796-8047-a7a18094875d None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:10.233053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44167ce8-6728-4796-8047-a7a18094875d None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:10.233688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44167ce8-6728-4796-8047-a7a18094875d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:10.241076 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 803/1609] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:10 2022] PUT /identity/v3/projects/2a4ddecea1cd4714a724ffe5c1231e04/users/0bd4a401c3ae43f5b6fe0a5468af3ebb/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:10.243741 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedaca27-f2df-4d1c-a1a8-223855bd04ad None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:10.244046 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedaca27-f2df-4d1c-a1a8-223855bd04ad None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:10.244326 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedaca27-f2df-4d1c-a1a8-223855bd04ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:10.263395 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-44167ce8-6728-4796-8047-a7a18094875d None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:10.264848 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-44167ce8-6728-4796-8047-a7a18094875d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:10.281584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-bedaca27-f2df-4d1c-a1a8-223855bd04ad None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:10.282621 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-bedaca27-f2df-4d1c-a1a8-223855bd04ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:10.336400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 807/1610] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:10.336400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f89c3-40a2-47dc-9036-bf522b0ae826 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:10.336400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f89c3-40a2-47dc-9036-bf522b0ae826 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:10.336400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f89c3-40a2-47dc-9036-bf522b0ae826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:10.351392 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 804/1611] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:10.352311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedaca27-f2df-4d1c-a1a8-223855bd04ad None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:10.354981 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-c97f89c3-40a2-47dc-9036-bf522b0ae826 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:10.356178 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-c97f89c3-40a2-47dc-9036-bf522b0ae826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:10.361967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedaca27-f2df-4d1c-a1a8-223855bd04ad None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:10.365636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 808/1612] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:10.366449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f89c3-40a2-47dc-9036-bf522b0ae826 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:10.367168 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53a1c3-64a4-4d5b-9c29-2d4e4a41ab80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:10.368079 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53a1c3-64a4-4d5b-9c29-2d4e4a41ab80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:10.368373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53a1c3-64a4-4d5b-9c29-2d4e4a41ab80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:10.369081 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53a1c3-64a4-4d5b-9c29-2d4e4a41ab80 admin admin] PATH_INFO: `/v3/projects/738dd11b726e4a25b22c7293c48f7a9a/users/d87972aae1b34ef7b46e8d7b6a50ba2a/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:10.385334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53a1c3-64a4-4d5b-9c29-2d4e4a41ab80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=738dd11b726e4a25b22c7293c48f7a9a, user_id=d87972aae1b34ef7b46e8d7b6a50ba2a, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:10.387999 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53a1c3-64a4-4d5b-9c29-2d4e4a41ab80 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:10.397391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:10.403797 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 805/1613] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:10 2022] PUT /identity/v3/projects/738dd11b726e4a25b22c7293c48f7a9a/users/d87972aae1b34ef7b46e8d7b6a50ba2a/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:10.411307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82077994-b609-4c57-b960-b215787f86c7 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:10.411791 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82077994-b609-4c57-b960-b215787f86c7 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:10.412033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82077994-b609-4c57-b960-b215787f86c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:10.429937 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f89c3-40a2-47dc-9036-bf522b0ae826 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:10.441757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-82077994-b609-4c57-b960-b215787f86c7 None None] MFA Rules not processed for user `0bd4a401c3ae43f5b6fe0a5468af3ebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:10.443141 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-82077994-b609-4c57-b960-b215787f86c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:10.485475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf355e8-e176-4300-8706-1288f9f3cb66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:10.486251 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf355e8-e176-4300-8706-1288f9f3cb66 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:10.486547 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf355e8-e176-4300-8706-1288f9f3cb66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:10.486839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf355e8-e176-4300-8706-1288f9f3cb66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:10.507714 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 806/1614] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:10 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:10.509054 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ae2ab-48b7-4897-97e6-0b2cd557786f None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:10.509380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ae2ab-48b7-4897-97e6-0b2cd557786f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:10.509666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ae2ab-48b7-4897-97e6-0b2cd557786f None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:10.511267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 807/1615] 10.222.0.40 () {54 vars in 912 bytes} [Thu Jun 16 17:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:34:10.511985 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ae2ab-48b7-4897-97e6-0b2cd557786f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:10.518381 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:10.523871 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ae2ab-48b7-4897-97e6-0b2cd557786f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:10.527561 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdcc252-3841-4352-ac10-acb0f68d3fbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:10.528210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcc252-3841-4352-ac10-acb0f68d3fbb service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:10.528501 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcc252-3841-4352-ac10-acb0f68d3fbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:10.528792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcc252-3841-4352-ac10-acb0f68d3fbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:10.553007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf355e8-e176-4300-8706-1288f9f3cb66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:10.554529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf355e8-e176-4300-8706-1288f9f3cb66 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:10.564412 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 809/1616] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:10.565352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf355e8-e176-4300-8706-1288f9f3cb66 service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:10.574614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf355e8-e176-4300-8706-1288f9f3cb66 service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:10.578607 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a60a5-08f6-4cf4-92b8-7346bcd75b7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:10.579260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a60a5-08f6-4cf4-92b8-7346bcd75b7c service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:10.579552 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a60a5-08f6-4cf4-92b8-7346bcd75b7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:10.579835 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a60a5-08f6-4cf4-92b8-7346bcd75b7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:10.591417 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a60a5-08f6-4cf4-92b8-7346bcd75b7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:10.592712 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a60a5-08f6-4cf4-92b8-7346bcd75b7c service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:10.593865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdcc252-3841-4352-ac10-acb0f68d3fbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:10.595300 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdcc252-3841-4352-ac10-acb0f68d3fbb service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:10.604666 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 810/1617] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:10.605622 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a60a5-08f6-4cf4-92b8-7346bcd75b7c service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:10.606439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 808/1618] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:10.607738 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74c3e6-01b5-4af9-9b94-cacf9901b40f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:10.608031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74c3e6-01b5-4af9-9b94-cacf9901b40f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:10.608312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74c3e6-01b5-4af9-9b94-cacf9901b40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:10.614379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a60a5-08f6-4cf4-92b8-7346bcd75b7c service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:10.619638 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ac31a-016d-4532-804d-529af50816c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:10.620296 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ac31a-016d-4532-804d-529af50816c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:10.620584 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ac31a-016d-4532-804d-529af50816c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:10.620881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ac31a-016d-4532-804d-529af50816c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:10.634399 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ac31a-016d-4532-804d-529af50816c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:10.636588 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ac31a-016d-4532-804d-529af50816c3 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:10.638937 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-3f74c3e6-01b5-4af9-9b94-cacf9901b40f None None] MFA Rules not processed for user `d87972aae1b34ef7b46e8d7b6a50ba2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:10.640535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3f74c3e6-01b5-4af9-9b94-cacf9901b40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:10.651312 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 811/1619] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:10.652300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ac31a-016d-4532-804d-529af50816c3 service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:10.666970 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ac31a-016d-4532-804d-529af50816c3 service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:10.678836 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbafdc0-97cb-4e33-b69a-4a5dff5fdf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:10.680327 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbafdc0-97cb-4e33-b69a-4a5dff5fdf96 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:10.680779 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbafdc0-97cb-4e33-b69a-4a5dff5fdf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:10.681297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbafdc0-97cb-4e33-b69a-4a5dff5fdf96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:10.682106 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbafdc0-97cb-4e33-b69a-4a5dff5fdf96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:10.685421 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbafdc0-97cb-4e33-b69a-4a5dff5fdf96 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:10.733949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 812/1620] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:10 2022] POST /identity/v3/projects => generated 363 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:10.735628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52dd3e-ca17-4120-b664-083be2f3ee44 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:10.735927 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52dd3e-ca17-4120-b664-083be2f3ee44 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:10.738101 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52dd3e-ca17-4120-b664-083be2f3ee44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:10.745967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 809/1621] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:10 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:10.746935 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74c3e6-01b5-4af9-9b94-cacf9901b40f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:10.767215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74c3e6-01b5-4af9-9b94-cacf9901b40f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:10.767324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-2c52dd3e-ca17-4120-b664-083be2f3ee44 None None] MFA Rules not processed for user `689a31e26c6b454a91c3fd79678b4d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:10.767324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-2c52dd3e-ca17-4120-b664-083be2f3ee44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:10.776890 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:10.851531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060d857-cf5a-4700-b19a-ef4dc0412393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:10.852278 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060d857-cf5a-4700-b19a-ef4dc0412393 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:10.852817 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060d857-cf5a-4700-b19a-ef4dc0412393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:10.853259 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060d857-cf5a-4700-b19a-ef4dc0412393 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:10.854957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060d857-cf5a-4700-b19a-ef4dc0412393 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:10.857499 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060d857-cf5a-4700-b19a-ef4dc0412393 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:10.871300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 813/1622] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 17:34:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:10.873165 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52dd3e-ca17-4120-b664-083be2f3ee44 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:10.886371 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 810/1623] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:10 2022] POST /identity/v3/users => generated 288 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:10.888023 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060d857-cf5a-4700-b19a-ef4dc0412393 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:10.902732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52dd3e-ca17-4120-b664-083be2f3ee44 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:10.911223 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab3d78-86b1-4fff-b2c1-8d49102d8223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:10.912299 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab3d78-86b1-4fff-b2c1-8d49102d8223 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:10.912846 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab3d78-86b1-4fff-b2c1-8d49102d8223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:10.913499 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab3d78-86b1-4fff-b2c1-8d49102d8223 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:10.915290 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab3d78-86b1-4fff-b2c1-8d49102d8223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:10.918216 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab3d78-86b1-4fff-b2c1-8d49102d8223 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:10.958067 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 814/1624] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:10 2022] POST /identity/v3/projects => generated 405 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:10.959380 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab3d78-86b1-4fff-b2c1-8d49102d8223 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:10.968425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060d857-cf5a-4700-b19a-ef4dc0412393 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:11.022187 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:11.042288 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab3d78-86b1-4fff-b2c1-8d49102d8223 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:11.052894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63353561-ced3-45b9-8d6b-d6a1d06c8a20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:11.060441 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63353561-ced3-45b9-8d6b-d6a1d06c8a20 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:11.060441 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63353561-ced3-45b9-8d6b-d6a1d06c8a20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:11.060441 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63353561-ced3-45b9-8d6b-d6a1d06c8a20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:11.066143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86258ff8-98d1-4f4b-980f-4ba63293ec67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:11.067991 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86258ff8-98d1-4f4b-980f-4ba63293ec67 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:11.068538 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86258ff8-98d1-4f4b-980f-4ba63293ec67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:11.069112 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86258ff8-98d1-4f4b-980f-4ba63293ec67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:11.070196 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86258ff8-98d1-4f4b-980f-4ba63293ec67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:11.074532 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86258ff8-98d1-4f4b-980f-4ba63293ec67 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:11.089785 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 815/1625] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:11.090602 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86258ff8-98d1-4f4b-980f-4ba63293ec67 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:11.105872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86258ff8-98d1-4f4b-980f-4ba63293ec67 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:11.112192 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23645eea-0663-4e63-9a3e-35fbe9de0a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:11.114903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23645eea-0663-4e63-9a3e-35fbe9de0a04 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:11.115342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23645eea-0663-4e63-9a3e-35fbe9de0a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:11.115731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23645eea-0663-4e63-9a3e-35fbe9de0a04 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:11.116647 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23645eea-0663-4e63-9a3e-35fbe9de0a04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:11.119517 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23645eea-0663-4e63-9a3e-35fbe9de0a04 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:11.158460 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 816/1626] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:11 2022] POST /identity/v3/users => generated 309 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:11.161007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23645eea-0663-4e63-9a3e-35fbe9de0a04 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:11.182541 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23645eea-0663-4e63-9a3e-35fbe9de0a04 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:11.191687 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfafee6-05b8-4bb7-9f40-20311fa1b614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:11.192838 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfafee6-05b8-4bb7-9f40-20311fa1b614 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:11.193242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfafee6-05b8-4bb7-9f40-20311fa1b614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:11.193608 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfafee6-05b8-4bb7-9f40-20311fa1b614 admin admin] PATH_INFO: `/v3/projects/bde6161bc2d74d24b4a4ab0c0d4de7ba/users/1016e143d9f4473c9ce7b5c1c01ae743/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:11.198218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63353561-ced3-45b9-8d6b-d6a1d06c8a20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:11.201065 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63353561-ced3-45b9-8d6b-d6a1d06c8a20 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:11.208915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:11.217172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfafee6-05b8-4bb7-9f40-20311fa1b614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde6161bc2d74d24b4a4ab0c0d4de7ba, user_id=1016e143d9f4473c9ce7b5c1c01ae743, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:11.224574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfafee6-05b8-4bb7-9f40-20311fa1b614 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:11.248875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 817/1627] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:11 2022] PUT /identity/v3/projects/bde6161bc2d74d24b4a4ab0c0d4de7ba/users/1016e143d9f4473c9ce7b5c1c01ae743/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:11.249576 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 811/1628] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:11.249853 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfafee6-05b8-4bb7-9f40-20311fa1b614 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:11.253395 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63353561-ced3-45b9-8d6b-d6a1d06c8a20 service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:11.298387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63353561-ced3-45b9-8d6b-d6a1d06c8a20 service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:11.432469 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfafee6-05b8-4bb7-9f40-20311fa1b614 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:11.451146 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377174c-f50b-4bda-a76e-c06ca00add76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:11.453985 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377174c-f50b-4bda-a76e-c06ca00add76 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:11.454276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377174c-f50b-4bda-a76e-c06ca00add76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:11.454575 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377174c-f50b-4bda-a76e-c06ca00add76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:11.455355 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377174c-f50b-4bda-a76e-c06ca00add76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:11.468659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123bd23-ca2b-4a25-92ae-23612e719812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:11.469461 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123bd23-ca2b-4a25-92ae-23612e719812 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:11.469765 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123bd23-ca2b-4a25-92ae-23612e719812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:11.470056 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123bd23-ca2b-4a25-92ae-23612e719812 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:11.470812 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123bd23-ca2b-4a25-92ae-23612e719812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:11.473681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123bd23-ca2b-4a25-92ae-23612e719812 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:11.475339 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377174c-f50b-4bda-a76e-c06ca00add76 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:11.492894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 818/1629] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:11.505200 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377174c-f50b-4bda-a76e-c06ca00add76 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:11.536613 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 812/1630] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:11.537935 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123bd23-ca2b-4a25-92ae-23612e719812 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:11.545909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377174c-f50b-4bda-a76e-c06ca00add76 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:11.565737 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6eff7-8115-4a28-9015-7d278acbedf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:11.572187 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123bd23-ca2b-4a25-92ae-23612e719812 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:11.573414 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6eff7-8115-4a28-9015-7d278acbedf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:11.575186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6eff7-8115-4a28-9015-7d278acbedf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:11.575186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6eff7-8115-4a28-9015-7d278acbedf9 admin admin] PATH_INFO: `/v3/projects/23a651fffaca4904bcbb34ce55867d75/users/7d99b3d4720c4c53ad87f749f749495f/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:11.595079 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b050d99-9e8c-43b7-a79b-e495de842e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:11.596005 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b050d99-9e8c-43b7-a79b-e495de842e90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:11.596295 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b050d99-9e8c-43b7-a79b-e495de842e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:11.596582 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b050d99-9e8c-43b7-a79b-e495de842e90 admin admin] PATH_INFO: `/v3/projects/bde6161bc2d74d24b4a4ab0c0d4de7ba/users/1016e143d9f4473c9ce7b5c1c01ae743/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:11.608453 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b050d99-9e8c-43b7-a79b-e495de842e90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde6161bc2d74d24b4a4ab0c0d4de7ba, user_id=1016e143d9f4473c9ce7b5c1c01ae743, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:11.617793 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b050d99-9e8c-43b7-a79b-e495de842e90 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:11.637042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6eff7-8115-4a28-9015-7d278acbedf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23a651fffaca4904bcbb34ce55867d75, user_id=7d99b3d4720c4c53ad87f749f749495f, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:11.641698 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6eff7-8115-4a28-9015-7d278acbedf9 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:11.675150 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 813/1631] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:11 2022] PUT /identity/v3/projects/bde6161bc2d74d24b4a4ab0c0d4de7ba/users/1016e143d9f4473c9ce7b5c1c01ae743/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:11.677055 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b609394-fac9-4b6d-9fe5-d794c4f1d24c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:11.677382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b609394-fac9-4b6d-9fe5-d794c4f1d24c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:11.677666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b609394-fac9-4b6d-9fe5-d794c4f1d24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:11.702098 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 819/1632] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:11 2022] PUT /identity/v3/projects/23a651fffaca4904bcbb34ce55867d75/users/7d99b3d4720c4c53ad87f749f749495f/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:11.703042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6eff7-8115-4a28-9015-7d278acbedf9 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:11.763635 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4b609394-fac9-4b6d-9fe5-d794c4f1d24c None None] MFA Rules not processed for user `1016e143d9f4473c9ce7b5c1c01ae743`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:11.764955 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4b609394-fac9-4b6d-9fe5-d794c4f1d24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:11.844578 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6eff7-8115-4a28-9015-7d278acbedf9 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:11.859086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03879456-bebf-4338-a819-05378dd10f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:11.859623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03879456-bebf-4338-a819-05378dd10f75 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:11.860288 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03879456-bebf-4338-a819-05378dd10f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:11.860404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03879456-bebf-4338-a819-05378dd10f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:11.861121 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03879456-bebf-4338-a819-05378dd10f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:11.869613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03879456-bebf-4338-a819-05378dd10f75 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:11.889089 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 820/1633] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:34:11.890676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d94a31-a1cb-4fed-9619-5b3de1584db7 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:11.890979 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d94a31-a1cb-4fed-9619-5b3de1584db7 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:11.891274 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d94a31-a1cb-4fed-9619-5b3de1584db7 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:11.892693 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 821/1634] 10.222.0.40 () {54 vars in 961 bytes} [Thu Jun 16 17:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:34:11.898793 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d94a31-a1cb-4fed-9619-5b3de1584db7 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:11.915968 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d94a31-a1cb-4fed-9619-5b3de1584db7 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:11.922092 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68a605-d6b1-4aa1-8a0b-46c0c58d4283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:11.923475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68a605-d6b1-4aa1-8a0b-46c0c58d4283 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:11.924212 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68a605-d6b1-4aa1-8a0b-46c0c58d4283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:11.924212 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68a605-d6b1-4aa1-8a0b-46c0c58d4283 admin admin] PATH_INFO: `/v3/projects/23a651fffaca4904bcbb34ce55867d75/users/7d99b3d4720c4c53ad87f749f749495f/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:11.927051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68a605-d6b1-4aa1-8a0b-46c0c58d4283 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23a651fffaca4904bcbb34ce55867d75, user_id=7d99b3d4720c4c53ad87f749f749495f, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:11.932878 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68a605-d6b1-4aa1-8a0b-46c0c58d4283 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:11.934827 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:11.947820 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 814/1636] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:11 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:11.950265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd01e1-182c-4bdb-acc3-25dbce322bbc None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:11.951722 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd01e1-182c-4bdb-acc3-25dbce322bbc None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:11.952017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd01e1-182c-4bdb-acc3-25dbce322bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:11.962008 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 822/1636] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:11 2022] PUT /identity/v3/projects/23a651fffaca4904bcbb34ce55867d75/users/7d99b3d4720c4c53ad87f749f749495f/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:11.963621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723a207-e925-4b01-8e37-975573f8fed8 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:11.963918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723a207-e925-4b01-8e37-975573f8fed8 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:11.964391 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723a207-e925-4b01-8e37-975573f8fed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:11.998245 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-d723a207-e925-4b01-8e37-975573f8fed8 None None] MFA Rules not processed for user `7d99b3d4720c4c53ad87f749f749495f`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:11.999468 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-dafd01e1-182c-4bdb-acc3-25dbce322bbc None None] MFA Rules not processed for user `689a31e26c6b454a91c3fd79678b4d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:11.999725 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-d723a207-e925-4b01-8e37-975573f8fed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:12.000754 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-dafd01e1-182c-4bdb-acc3-25dbce322bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:12.088464 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:12.093572 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 823/1637] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:11 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:12.096953 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723a207-e925-4b01-8e37-975573f8fed8 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:12.120167 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 815/1638] 10.222.0.40 () {58 vars in 1058 bytes} [Thu Jun 16 17:34:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:12.122727 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472aac8-ac82-4437-a691-680554323ed4 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:12.123654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472aac8-ac82-4437-a691-680554323ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:12.124236 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472aac8-ac82-4437-a691-680554323ed4 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:12.133221 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 816/1639] 10.222.0.40 () {54 vars in 912 bytes} [Thu Jun 16 17:34:12 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:34:12.135001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a8c11-860b-4e0e-9f94-4b36d1739649 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:12.135820 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a8c11-860b-4e0e-9f94-4b36d1739649 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:12.136902 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a8c11-860b-4e0e-9f94-4b36d1739649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:12.162401 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-485a8c11-860b-4e0e-9f94-4b36d1739649 None None] MFA Rules not processed for user `689a31e26c6b454a91c3fd79678b4d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:12.164162 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-485a8c11-860b-4e0e-9f94-4b36d1739649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:12.184716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 817/1640] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 17:34:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:12.186122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a8c11-860b-4e0e-9f94-4b36d1739649 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:12.246400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723a207-e925-4b01-8e37-975573f8fed8 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:12.258881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd604ec9-4c2a-4e24-a119-b28b6342078d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:12.259744 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd604ec9-4c2a-4e24-a119-b28b6342078d service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:12.260025 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd604ec9-4c2a-4e24-a119-b28b6342078d service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:12.260320 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd604ec9-4c2a-4e24-a119-b28b6342078d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:12.308886 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a8c11-860b-4e0e-9f94-4b36d1739649 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:12.324178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dacbdb2-00c7-4555-924d-1ecfb2f898c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:12.325344 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacbdb2-00c7-4555-924d-1ecfb2f898c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:12.328636 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacbdb2-00c7-4555-924d-1ecfb2f898c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:12.328966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacbdb2-00c7-4555-924d-1ecfb2f898c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:12.366934 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dacbdb2-00c7-4555-924d-1ecfb2f898c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:12.370399 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dacbdb2-00c7-4555-924d-1ecfb2f898c8 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:12.406916 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 818/1641] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:12 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:12.466483 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd604ec9-4c2a-4e24-a119-b28b6342078d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:12.469038 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd604ec9-4c2a-4e24-a119-b28b6342078d service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:12.509195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 824/1642] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:12 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:12.584729 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dacbdb2-00c7-4555-924d-1ecfb2f898c8 service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:12.601670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dacbdb2-00c7-4555-924d-1ecfb2f898c8 service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:12.617073 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c476f7d-211f-47b4-bd0f-c4772a42c5af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:12.619037 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c476f7d-211f-47b4-bd0f-c4772a42c5af service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:12.619586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c476f7d-211f-47b4-bd0f-c4772a42c5af service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:12.621624 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c476f7d-211f-47b4-bd0f-c4772a42c5af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:12.654458 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c476f7d-211f-47b4-bd0f-c4772a42c5af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:12.657003 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c476f7d-211f-47b4-bd0f-c4772a42c5af service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:12.686706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 819/1643] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:12.811817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd604ec9-4c2a-4e24-a119-b28b6342078d service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:12.823705 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd604ec9-4c2a-4e24-a119-b28b6342078d service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:12.851373 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0320f6-561e-4284-96d4-9ebef5c09b67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:12.852847 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0320f6-561e-4284-96d4-9ebef5c09b67 service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:12.854271 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0320f6-561e-4284-96d4-9ebef5c09b67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:12.855286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0320f6-561e-4284-96d4-9ebef5c09b67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:12.946208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:13.004084 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0320f6-561e-4284-96d4-9ebef5c09b67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:13.005972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0320f6-561e-4284-96d4-9ebef5c09b67 service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:13.031500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 825/1644] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:13.329232 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c476f7d-211f-47b4-bd0f-c4772a42c5af service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:13.601129 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c476f7d-211f-47b4-bd0f-c4772a42c5af service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:13.618544 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110964c9-68db-42b6-a7ef-e035b355fcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:13.620474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110964c9-68db-42b6-a7ef-e035b355fcea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:13.620740 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110964c9-68db-42b6-a7ef-e035b355fcea admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:13.626438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110964c9-68db-42b6-a7ef-e035b355fcea admin admin] PATH_INFO: `/v3/users/d87972aae1b34ef7b46e8d7b6a50ba2a` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:13.627948 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110964c9-68db-42b6-a7ef-e035b355fcea admin admin] RBAC: Authorizing `identity:delete_user(user_id=d87972aae1b34ef7b46e8d7b6a50ba2a)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:13.637499 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110964c9-68db-42b6-a7ef-e035b355fcea admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:13.726676 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-110964c9-68db-42b6-a7ef-e035b355fcea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd87972aae1b34ef7b46e8d7b6a50ba2a'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:13.757560 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-110964c9-68db-42b6-a7ef-e035b355fcea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd87972aae1b34ef7b46e8d7b6a50ba2a'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:13.758920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-110964c9-68db-42b6-a7ef-e035b355fcea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd87972aae1b34ef7b46e8d7b6a50ba2a'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:13.795961 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-110964c9-68db-42b6-a7ef-e035b355fcea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd87972aae1b34ef7b46e8d7b6a50ba2a'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:13.813577 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-110964c9-68db-42b6-a7ef-e035b355fcea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd87972aae1b34ef7b46e8d7b6a50ba2a'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:13.819254 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:13.830726 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b07de4-46ba-4f8b-9f6c-9b43e27a08c6 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:13.831032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b07de4-46ba-4f8b-9f6c-9b43e27a08c6 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:13.831313 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b07de4-46ba-4f8b-9f6c-9b43e27a08c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:13.834814 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 820/1645] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:34:13 2022] DELETE /identity/v3/users/d87972aae1b34ef7b46e8d7b6a50ba2a => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:13.864502 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-a3b07de4-46ba-4f8b-9f6c-9b43e27a08c6 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:13.866762 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-a3b07de4-46ba-4f8b-9f6c-9b43e27a08c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:13.940549 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 826/1646] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:34:13.962241 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110964c9-68db-42b6-a7ef-e035b355fcea admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:13.963728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-110964c9-68db-42b6-a7ef-e035b355fcea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:14.096076 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110964c9-68db-42b6-a7ef-e035b355fcea admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:14.173076 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6915ef-4ebc-4580-b92c-68752d4c87e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:14.173076 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6915ef-4ebc-4580-b92c-68752d4c87e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:14.173076 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6915ef-4ebc-4580-b92c-68752d4c87e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:14.173076 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6915ef-4ebc-4580-b92c-68752d4c87e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:14.204689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6915ef-4ebc-4580-b92c-68752d4c87e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:14.204689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6915ef-4ebc-4580-b92c-68752d4c87e5 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:14.204689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 821/1647] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:14.743520 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b07de4-46ba-4f8b-9f6c-9b43e27a08c6 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:14.746583 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-a3b07de4-46ba-4f8b-9f6c-9b43e27a08c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:14.770626 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b07de4-46ba-4f8b-9f6c-9b43e27a08c6 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:14.791005 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59928d-013f-495b-8b22-6d533dd9a77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:14.791929 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59928d-013f-495b-8b22-6d533dd9a77d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:14.792257 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59928d-013f-495b-8b22-6d533dd9a77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:14.792884 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59928d-013f-495b-8b22-6d533dd9a77d admin admin] PATH_INFO: `/v3/users/09185fe6124c4663b2f974616a8120b8` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:14.794206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59928d-013f-495b-8b22-6d533dd9a77d admin admin] RBAC: Authorizing `identity:delete_user(user_id=09185fe6124c4663b2f974616a8120b8)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:14.795257 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: WARNING py.warnings [None req-bb59928d-013f-495b-8b22-6d533dd9a77d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:34:14.795257 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: warnings.warn(msg) Jun 16 17:34:14.795257 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: Jun 16 17:34:14.796501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59928d-013f-495b-8b22-6d533dd9a77d admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:14.990603 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-bb59928d-013f-495b-8b22-6d533dd9a77d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09185fe6124c4663b2f974616a8120b8'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:15.019447 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-bb59928d-013f-495b-8b22-6d533dd9a77d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09185fe6124c4663b2f974616a8120b8'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:15.020567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-bb59928d-013f-495b-8b22-6d533dd9a77d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09185fe6124c4663b2f974616a8120b8'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:15.052394 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-bb59928d-013f-495b-8b22-6d533dd9a77d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09185fe6124c4663b2f974616a8120b8'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:15.127571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-bb59928d-013f-495b-8b22-6d533dd9a77d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09185fe6124c4663b2f974616a8120b8'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:15.165664 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 827/1648] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:34:14 2022] DELETE /identity/v3/users/09185fe6124c4663b2f974616a8120b8 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:15.167399 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c558f7-6b5d-453a-9d5a-42b4d35a6bdb None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:15.167916 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c558f7-6b5d-453a-9d5a-42b4d35a6bdb None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:15.168018 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c558f7-6b5d-453a-9d5a-42b4d35a6bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:15.213782 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a0c558f7-6b5d-453a-9d5a-42b4d35a6bdb None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:15.213782 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a0c558f7-6b5d-453a-9d5a-42b4d35a6bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:15.407929 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 822/1649] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:15.455889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59928d-013f-495b-8b22-6d533dd9a77d admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:15.458648 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-bb59928d-013f-495b-8b22-6d533dd9a77d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:15.567833 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:15.609580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59928d-013f-495b-8b22-6d533dd9a77d admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:15.632992 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958af52-9f55-4213-99e4-91d2ee070991 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:15.633653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958af52-9f55-4213-99e4-91d2ee070991 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:15.633946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958af52-9f55-4213-99e4-91d2ee070991 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:15.634230 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958af52-9f55-4213-99e4-91d2ee070991 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:15.669350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958af52-9f55-4213-99e4-91d2ee070991 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:15.674308 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958af52-9f55-4213-99e4-91d2ee070991 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:15.702332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 828/1650] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:16.245858 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c558f7-6b5d-453a-9d5a-42b4d35a6bdb None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:16.249991 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a0c558f7-6b5d-453a-9d5a-42b4d35a6bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:16.279578 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c558f7-6b5d-453a-9d5a-42b4d35a6bdb None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:16.293881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53e933-63a3-4606-9838-55f15571a15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:16.294789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53e933-63a3-4606-9838-55f15571a15e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:16.295077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53e933-63a3-4606-9838-55f15571a15e admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:16.295367 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53e933-63a3-4606-9838-55f15571a15e admin admin] PATH_INFO: `/v3/projects/738dd11b726e4a25b22c7293c48f7a9a` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:16.296630 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53e933-63a3-4606-9838-55f15571a15e admin admin] RBAC: Authorizing `identity:delete_project(project_id=738dd11b726e4a25b22c7293c48f7a9a)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:16.297711 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: WARNING py.warnings [None req-8c53e933-63a3-4606-9838-55f15571a15e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:34:16.297711 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: warnings.warn(msg) Jun 16 17:34:16.297711 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: Jun 16 17:34:16.298926 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53e933-63a3-4606-9838-55f15571a15e admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:16.429109 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8c53e933-63a3-4606-9838-55f15571a15e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '738dd11b726e4a25b22c7293c48f7a9a', 'request_id': 'req-8c53e933-63a3-4606-9838-55f15571a15e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:16.444724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8c53e933-63a3-4606-9838-55f15571a15e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '738dd11b726e4a25b22c7293c48f7a9a', 'request_id': 'req-8c53e933-63a3-4606-9838-55f15571a15e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:16.463377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8c53e933-63a3-4606-9838-55f15571a15e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '738dd11b726e4a25b22c7293c48f7a9a', 'request_id': 'req-8c53e933-63a3-4606-9838-55f15571a15e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:16.477552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8c53e933-63a3-4606-9838-55f15571a15e admin admin] The token cache is being invalidate because project 738dd11b726e4a25b22c7293c48f7a9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:34:16.477826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8c53e933-63a3-4606-9838-55f15571a15e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:16.478516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8c53e933-63a3-4606-9838-55f15571a15e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:16.483072 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 823/1651] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:34:16 2022] DELETE /identity/v3/projects/738dd11b726e4a25b22c7293c48f7a9a => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:16.484248 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958af52-9f55-4213-99e4-91d2ee070991 service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:16.499505 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958af52-9f55-4213-99e4-91d2ee070991 service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:16.692292 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54979b49-835a-42b2-ae95-075ccef5b8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:16.693350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54979b49-835a-42b2-ae95-075ccef5b8be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:16.695720 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54979b49-835a-42b2-ae95-075ccef5b8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:16.696030 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54979b49-835a-42b2-ae95-075ccef5b8be admin admin] PATH_INFO: `/v3/users/7d99b3d4720c4c53ad87f749f749495f` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:16.713408 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54979b49-835a-42b2-ae95-075ccef5b8be admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d99b3d4720c4c53ad87f749f749495f)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:16.715438 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54979b49-835a-42b2-ae95-075ccef5b8be admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:16.888179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-54979b49-835a-42b2-ae95-075ccef5b8be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d99b3d4720c4c53ad87f749f749495f'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:16.920181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-54979b49-835a-42b2-ae95-075ccef5b8be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d99b3d4720c4c53ad87f749f749495f'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:16.920992 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-54979b49-835a-42b2-ae95-075ccef5b8be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d99b3d4720c4c53ad87f749f749495f'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:16.933357 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-54979b49-835a-42b2-ae95-075ccef5b8be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d99b3d4720c4c53ad87f749f749495f'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:16.999445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-54979b49-835a-42b2-ae95-075ccef5b8be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d99b3d4720c4c53ad87f749f749495f'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:17.039498 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 829/1652] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:34:16 2022] DELETE /identity/v3/users/7d99b3d4720c4c53ad87f749f749495f => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:17.090734 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53e933-63a3-4606-9838-55f15571a15e admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:17.092172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8c53e933-63a3-4606-9838-55f15571a15e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:17.114628 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53e933-63a3-4606-9838-55f15571a15e admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:17.186795 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:17.324101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cfd65-67e2-425e-9b38-23f26950c119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:17.338450 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cfd65-67e2-425e-9b38-23f26950c119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:17.342742 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cfd65-67e2-425e-9b38-23f26950c119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:17.343078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cfd65-67e2-425e-9b38-23f26950c119 admin admin] PATH_INFO: `/v3/users/0bd4a401c3ae43f5b6fe0a5468af3ebb` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:17.351575 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5cfd65-67e2-425e-9b38-23f26950c119 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bd4a401c3ae43f5b6fe0a5468af3ebb)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:17.359840 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5cfd65-67e2-425e-9b38-23f26950c119 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:17.407710 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54979b49-835a-42b2-ae95-075ccef5b8be admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:17.409214 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-54979b49-835a-42b2-ae95-075ccef5b8be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:17.443249 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54979b49-835a-42b2-ae95-075ccef5b8be admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:17.470038 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e83160-7f07-48e8-8e4d-d35b51ea7e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:17.471139 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e83160-7f07-48e8-8e4d-d35b51ea7e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:17.471439 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e83160-7f07-48e8-8e4d-d35b51ea7e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:17.471730 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e83160-7f07-48e8-8e4d-d35b51ea7e21 admin admin] PATH_INFO: `/v3/projects/7dc9438583194d70b654dd8ce36417e7` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:17.481177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7d5cfd65-67e2-425e-9b38-23f26950c119 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bd4a401c3ae43f5b6fe0a5468af3ebb'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:17.481661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e83160-7f07-48e8-8e4d-d35b51ea7e21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dc9438583194d70b654dd8ce36417e7)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:17.491016 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e83160-7f07-48e8-8e4d-d35b51ea7e21 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:17.521015 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7d5cfd65-67e2-425e-9b38-23f26950c119 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bd4a401c3ae43f5b6fe0a5468af3ebb'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:17.522089 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7d5cfd65-67e2-425e-9b38-23f26950c119 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bd4a401c3ae43f5b6fe0a5468af3ebb'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:17.540813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7d5cfd65-67e2-425e-9b38-23f26950c119 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bd4a401c3ae43f5b6fe0a5468af3ebb'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:17.583587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-7d5cfd65-67e2-425e-9b38-23f26950c119 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bd4a401c3ae43f5b6fe0a5468af3ebb'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:17.593202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 824/1653] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:34:17 2022] DELETE /identity/v3/users/0bd4a401c3ae43f5b6fe0a5468af3ebb => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:17.598220 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbee70-5a51-4ff2-8e28-3fff49ae5b9a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:17.598818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbee70-5a51-4ff2-8e28-3fff49ae5b9a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:17.599322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbee70-5a51-4ff2-8e28-3fff49ae5b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:17.642601 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-57fbee70-5a51-4ff2-8e28-3fff49ae5b9a None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:17.644531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-57fbee70-5a51-4ff2-8e28-3fff49ae5b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:17.746408 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a2e83160-7f07-48e8-8e4d-d35b51ea7e21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dc9438583194d70b654dd8ce36417e7', 'request_id': 'req-a2e83160-7f07-48e8-8e4d-d35b51ea7e21'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:17.796549 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a2e83160-7f07-48e8-8e4d-d35b51ea7e21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dc9438583194d70b654dd8ce36417e7', 'request_id': 'req-a2e83160-7f07-48e8-8e4d-d35b51ea7e21'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:17.805804 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:17.809467 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 825/1654] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:34:17.830264 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a2e83160-7f07-48e8-8e4d-d35b51ea7e21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dc9438583194d70b654dd8ce36417e7', 'request_id': 'req-a2e83160-7f07-48e8-8e4d-d35b51ea7e21'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:17.860061 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a2e83160-7f07-48e8-8e4d-d35b51ea7e21 admin admin] The token cache is being invalidate because project 7dc9438583194d70b654dd8ce36417e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:34:17.860955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a2e83160-7f07-48e8-8e4d-d35b51ea7e21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:17.862891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a2e83160-7f07-48e8-8e4d-d35b51ea7e21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:17.864890 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 830/1655] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:34:17 2022] DELETE /identity/v3/projects/7dc9438583194d70b654dd8ce36417e7 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:17.870360 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe8654-0593-49c4-96c4-1ab077e8057e None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:17.870726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe8654-0593-49c4-96c4-1ab077e8057e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:17.871069 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe8654-0593-49c4-96c4-1ab077e8057e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:17.905067 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-7dfe8654-0593-49c4-96c4-1ab077e8057e None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:17.907200 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7dfe8654-0593-49c4-96c4-1ab077e8057e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:18.051293 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:18.113492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 826/1656] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:18.120856 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420549b8-ccc2-4a95-b9f0-989927d77520 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:18.121650 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420549b8-ccc2-4a95-b9f0-989927d77520 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:18.121797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420549b8-ccc2-4a95-b9f0-989927d77520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:18.162165 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-420549b8-ccc2-4a95-b9f0-989927d77520 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:18.167591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-420549b8-ccc2-4a95-b9f0-989927d77520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:18.210736 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe8654-0593-49c4-96c4-1ab077e8057e None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:18.212545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7dfe8654-0593-49c4-96c4-1ab077e8057e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:18.219121 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 831/1657] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:18.225220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420549b8-ccc2-4a95-b9f0-989927d77520 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:18.266324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420549b8-ccc2-4a95-b9f0-989927d77520 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:18.293412 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a763de-e5c0-4e0f-ba43-09789c8abb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:18.294436 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a763de-e5c0-4e0f-ba43-09789c8abb06 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:18.294861 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a763de-e5c0-4e0f-ba43-09789c8abb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:18.298078 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a763de-e5c0-4e0f-ba43-09789c8abb06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:18.299331 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a763de-e5c0-4e0f-ba43-09789c8abb06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:18.299791 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a763de-e5c0-4e0f-ba43-09789c8abb06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:18.308005 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a763de-e5c0-4e0f-ba43-09789c8abb06 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:18.364120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 832/1658] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:18.366839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a763de-e5c0-4e0f-ba43-09789c8abb06 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:18.406130 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a763de-e5c0-4e0f-ba43-09789c8abb06 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:18.420874 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2883d00-31ad-424c-8ba0-8a3778ebd89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:18.423218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2883d00-31ad-424c-8ba0-8a3778ebd89a admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:18.424537 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe8654-0593-49c4-96c4-1ab077e8057e None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:18.426131 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2883d00-31ad-424c-8ba0-8a3778ebd89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:18.426457 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2883d00-31ad-424c-8ba0-8a3778ebd89a admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:18.427267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2883d00-31ad-424c-8ba0-8a3778ebd89a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:18.429983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2883d00-31ad-424c-8ba0-8a3778ebd89a admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:18.435944 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745bc90-5ce7-4e47-91e3-5e5693c4493e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:18.438779 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745bc90-5ce7-4e47-91e3-5e5693c4493e service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:18.439164 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745bc90-5ce7-4e47-91e3-5e5693c4493e service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:18.439526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745bc90-5ce7-4e47-91e3-5e5693c4493e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:18.443101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9745bc90-5ce7-4e47-91e3-5e5693c4493e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:18.468799 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745bc90-5ce7-4e47-91e3-5e5693c4493e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:18.471529 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745bc90-5ce7-4e47-91e3-5e5693c4493e service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:18.492430 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 827/1659] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:34:18.502195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 833/1660] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:18 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:18.507082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745bc90-5ce7-4e47-91e3-5e5693c4493e service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:18.523473 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745bc90-5ce7-4e47-91e3-5e5693c4493e service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:18.661077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca14a35-1135-4d08-a82d-8eea453c1593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:18.661985 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca14a35-1135-4d08-a82d-8eea453c1593 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:18.662384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca14a35-1135-4d08-a82d-8eea453c1593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:18.662671 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca14a35-1135-4d08-a82d-8eea453c1593 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:18.663391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca14a35-1135-4d08-a82d-8eea453c1593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:18.665636 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca14a35-1135-4d08-a82d-8eea453c1593 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:18.713334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 828/1661] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:18 2022] POST /identity/v3/users => generated 287 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:18.717470 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2883d00-31ad-424c-8ba0-8a3778ebd89a admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:18.730686 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2883d00-31ad-424c-8ba0-8a3778ebd89a admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:18.744708 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f97301-738f-400d-83cb-5d7e8c3da072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:18.745677 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f97301-738f-400d-83cb-5d7e8c3da072 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:18.745972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f97301-738f-400d-83cb-5d7e8c3da072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:18.746254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f97301-738f-400d-83cb-5d7e8c3da072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:18.746999 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f97301-738f-400d-83cb-5d7e8c3da072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:18.752066 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f97301-738f-400d-83cb-5d7e8c3da072 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:18.782735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 834/1662] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:18.783947 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca14a35-1135-4d08-a82d-8eea453c1593 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:18.805958 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca14a35-1135-4d08-a82d-8eea453c1593 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:18.823390 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291cf88e-050f-46a1-90c0-135b3d0b43a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:18.824083 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cf88e-050f-46a1-90c0-135b3d0b43a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:18.824371 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cf88e-050f-46a1-90c0-135b3d0b43a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:18.824650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cf88e-050f-46a1-90c0-135b3d0b43a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:18.825424 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291cf88e-050f-46a1-90c0-135b3d0b43a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:18.827034 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291cf88e-050f-46a1-90c0-135b3d0b43a7 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:18.853574 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 829/1663] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:18.857662 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f97301-738f-400d-83cb-5d7e8c3da072 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:18.871774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f97301-738f-400d-83cb-5d7e8c3da072 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:18.889688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29810039-c5cf-45cf-940b-c559461df3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:18.890613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29810039-c5cf-45cf-940b-c559461df3c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:18.890898 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29810039-c5cf-45cf-940b-c559461df3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:18.891178 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29810039-c5cf-45cf-940b-c559461df3c7 admin admin] PATH_INFO: `/v3/projects/630ce45e2ddb42e582e52cd42f2920b2/users/1a858dbccddf49a285dafcc972a025cc/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:18.926236 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29810039-c5cf-45cf-940b-c559461df3c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=630ce45e2ddb42e582e52cd42f2920b2, user_id=1a858dbccddf49a285dafcc972a025cc, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:18.929278 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29810039-c5cf-45cf-940b-c559461df3c7 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:18.963250 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 835/1664] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:18 2022] PUT /identity/v3/projects/630ce45e2ddb42e582e52cd42f2920b2/users/1a858dbccddf49a285dafcc972a025cc/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:18.981388 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc313b-b07a-45c6-b3ac-30c8caa5b72c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:18.981688 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc313b-b07a-45c6-b3ac-30c8caa5b72c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:18.981970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc313b-b07a-45c6-b3ac-30c8caa5b72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:19.071004 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-98cc313b-b07a-45c6-b3ac-30c8caa5b72c None None] MFA Rules not processed for user `1a858dbccddf49a285dafcc972a025cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:19.072417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-98cc313b-b07a-45c6-b3ac-30c8caa5b72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:19.246493 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:19.287775 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29810039-c5cf-45cf-940b-c559461df3c7 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:19.290701 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 830/1665] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:19.525750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29810039-c5cf-45cf-940b-c559461df3c7 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:19.542626 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c15c4-45b8-4204-9d5a-c3f324ca159d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:19.543741 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c15c4-45b8-4204-9d5a-c3f324ca159d admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:19.544229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c15c4-45b8-4204-9d5a-c3f324ca159d admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:19.545975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c15c4-45b8-4204-9d5a-c3f324ca159d admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:19.547563 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c15c4-45b8-4204-9d5a-c3f324ca159d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:19.549856 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c15c4-45b8-4204-9d5a-c3f324ca159d admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:19.632522 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc313b-b07a-45c6-b3ac-30c8caa5b72c None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:19.648200 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 836/1666] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:19 2022] POST /identity/v3/projects => generated 369 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:19.652206 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc313b-b07a-45c6-b3ac-30c8caa5b72c None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:19.811679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44350173-2a0c-4700-b0b3-b5202b5e0cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:19.812482 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44350173-2a0c-4700-b0b3-b5202b5e0cad admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:19.812843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44350173-2a0c-4700-b0b3-b5202b5e0cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:19.813210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44350173-2a0c-4700-b0b3-b5202b5e0cad admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:19.814207 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44350173-2a0c-4700-b0b3-b5202b5e0cad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:19.818185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44350173-2a0c-4700-b0b3-b5202b5e0cad admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:19.883605 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c15c4-45b8-4204-9d5a-c3f324ca159d admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:19.897294 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 831/1667] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:19 2022] POST /identity/v3/users => generated 291 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:34:19.898271 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c15c4-45b8-4204-9d5a-c3f324ca159d admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:19.912395 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3caf3-2bd9-4c8f-a10a-9d3f03975637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:19.914455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3caf3-2bd9-4c8f-a10a-9d3f03975637 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:19.914869 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3caf3-2bd9-4c8f-a10a-9d3f03975637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:19.915274 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3caf3-2bd9-4c8f-a10a-9d3f03975637 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:19.916292 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f3caf3-2bd9-4c8f-a10a-9d3f03975637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:19.924668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f3caf3-2bd9-4c8f-a10a-9d3f03975637 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:19.954240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44350173-2a0c-4700-b0b3-b5202b5e0cad admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:19.970067 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44350173-2a0c-4700-b0b3-b5202b5e0cad admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:19.971013 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 837/1668] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:34:19.986169 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a02cc3-0b26-470e-85d7-6f37736bdb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:19.987256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a02cc3-0b26-470e-85d7-6f37736bdb06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:19.988297 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a02cc3-0b26-470e-85d7-6f37736bdb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:19.988680 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a02cc3-0b26-470e-85d7-6f37736bdb06 admin admin] PATH_INFO: `/v3/projects/7f74f72d8edd4bada30cb56b91a8dbbb/users/09ec37c663074430951d659cb60a0023/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:20.027903 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a02cc3-0b26-470e-85d7-6f37736bdb06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f74f72d8edd4bada30cb56b91a8dbbb, user_id=09ec37c663074430951d659cb60a0023, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:20.030781 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a02cc3-0b26-470e-85d7-6f37736bdb06 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:20.066148 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3caf3-2bd9-4c8f-a10a-9d3f03975637 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:20.066770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 832/1669] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:19 2022] PUT /identity/v3/projects/7f74f72d8edd4bada30cb56b91a8dbbb/users/09ec37c663074430951d659cb60a0023/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:20.079326 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3caf3-2bd9-4c8f-a10a-9d3f03975637 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:20.169788 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:20.191526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32502651-e533-40c5-88b5-28b5358441fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:20.192379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32502651-e533-40c5-88b5-28b5358441fd admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:20.193718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32502651-e533-40c5-88b5-28b5358441fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:20.194122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32502651-e533-40c5-88b5-28b5358441fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:20.195126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32502651-e533-40c5-88b5-28b5358441fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:20.196852 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32502651-e533-40c5-88b5-28b5358441fd admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:20.210539 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 838/1670] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:20.214677 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a02cc3-0b26-470e-85d7-6f37736bdb06 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:20.227531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a02cc3-0b26-470e-85d7-6f37736bdb06 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:20.233503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd8a44-5667-4dd6-916b-1e4b89017176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:20.234906 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd8a44-5667-4dd6-916b-1e4b89017176 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:20.235144 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd8a44-5667-4dd6-916b-1e4b89017176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:20.235585 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd8a44-5667-4dd6-916b-1e4b89017176 admin admin] PATH_INFO: `/v3/projects/7f74f72d8edd4bada30cb56b91a8dbbb/users/09ec37c663074430951d659cb60a0023/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:20.238714 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd8a44-5667-4dd6-916b-1e4b89017176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f74f72d8edd4bada30cb56b91a8dbbb, user_id=09ec37c663074430951d659cb60a0023, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:20.241619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd8a44-5667-4dd6-916b-1e4b89017176 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:20.267589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 833/1671] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:20 2022] PUT /identity/v3/projects/7f74f72d8edd4bada30cb56b91a8dbbb/users/09ec37c663074430951d659cb60a0023/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:20.269794 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16343c-f54c-4c73-a064-bc4c2be24351 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:20.270449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16343c-f54c-4c73-a064-bc4c2be24351 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:20.271055 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16343c-f54c-4c73-a064-bc4c2be24351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:20.326412 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-5b16343c-f54c-4c73-a064-bc4c2be24351 None None] MFA Rules not processed for user `09ec37c663074430951d659cb60a0023`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:20.327954 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-5b16343c-f54c-4c73-a064-bc4c2be24351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:20.611935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 839/1672] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:20 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:20.635219 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd8a44-5667-4dd6-916b-1e4b89017176 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:20.638012 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-dbfd8a44-5667-4dd6-916b-1e4b89017176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:20.747065 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd8a44-5667-4dd6-916b-1e4b89017176 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:20.751665 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e910c88f-7ac4-432d-a274-51811bd5843f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:20.753629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e910c88f-7ac4-432d-a274-51811bd5843f service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:20.754330 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e910c88f-7ac4-432d-a274-51811bd5843f service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:20.754854 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e910c88f-7ac4-432d-a274-51811bd5843f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:20.767460 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e910c88f-7ac4-432d-a274-51811bd5843f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:20.772900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e910c88f-7ac4-432d-a274-51811bd5843f service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:20.806813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 834/1673] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:20 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:20.941962 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b16343c-f54c-4c73-a064-bc4c2be24351 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:20.954026 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b16343c-f54c-4c73-a064-bc4c2be24351 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:20.959556 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea97275-9fa2-422e-adbc-718afcaca9fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:20.960709 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea97275-9fa2-422e-adbc-718afcaca9fc service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:20.961327 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea97275-9fa2-422e-adbc-718afcaca9fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:20.961805 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea97275-9fa2-422e-adbc-718afcaca9fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:21.148576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea97275-9fa2-422e-adbc-718afcaca9fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:21.150081 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea97275-9fa2-422e-adbc-718afcaca9fc service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:21.170495 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 840/1674] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:22.002552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e910c88f-7ac4-432d-a274-51811bd5843f service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:22.004167 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e910c88f-7ac4-432d-a274-51811bd5843f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:22.166929 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea97275-9fa2-422e-adbc-718afcaca9fc service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:22.171821 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-3ea97275-9fa2-422e-adbc-718afcaca9fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:22.174529 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:22.214523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea97275-9fa2-422e-adbc-718afcaca9fc service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:22.224639 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e910c88f-7ac4-432d-a274-51811bd5843f service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:22.243051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae1642-c578-4a87-8f9a-93ae81c8947c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:22.245582 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae1642-c578-4a87-8f9a-93ae81c8947c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:22.246019 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae1642-c578-4a87-8f9a-93ae81c8947c admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:22.247208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae1642-c578-4a87-8f9a-93ae81c8947c admin admin] PATH_INFO: `/v3/projects/2a4ddecea1cd4714a724ffe5c1231e04` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:22.250608 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ae1642-c578-4a87-8f9a-93ae81c8947c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a4ddecea1cd4714a724ffe5c1231e04)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:22.253700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8a0c7-05bc-40ef-8451-112091d09804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:22.254790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8a0c7-05bc-40ef-8451-112091d09804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:22.255224 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8a0c7-05bc-40ef-8451-112091d09804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:22.255650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8a0c7-05bc-40ef-8451-112091d09804 admin admin] PATH_INFO: `/v3/projects/23a651fffaca4904bcbb34ce55867d75` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:22.256202 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ae1642-c578-4a87-8f9a-93ae81c8947c admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:22.257353 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8a0c7-05bc-40ef-8451-112091d09804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23a651fffaca4904bcbb34ce55867d75)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:22.259438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8a0c7-05bc-40ef-8451-112091d09804 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:22.358924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-00b8a0c7-05bc-40ef-8451-112091d09804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23a651fffaca4904bcbb34ce55867d75', 'request_id': 'req-00b8a0c7-05bc-40ef-8451-112091d09804'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:22.374773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-00b8a0c7-05bc-40ef-8451-112091d09804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23a651fffaca4904bcbb34ce55867d75', 'request_id': 'req-00b8a0c7-05bc-40ef-8451-112091d09804'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:22.390235 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-00b8a0c7-05bc-40ef-8451-112091d09804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23a651fffaca4904bcbb34ce55867d75', 'request_id': 'req-00b8a0c7-05bc-40ef-8451-112091d09804'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:22.419653 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-00b8a0c7-05bc-40ef-8451-112091d09804 admin admin] The token cache is being invalidate because project 23a651fffaca4904bcbb34ce55867d75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:34:22.420161 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-00b8a0c7-05bc-40ef-8451-112091d09804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:22.421025 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-00b8a0c7-05bc-40ef-8451-112091d09804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:22.423039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 835/1675] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:34:22 2022] DELETE /identity/v3/projects/23a651fffaca4904bcbb34ce55867d75 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:22.428222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709850ac-362d-4e62-8b18-4897d24ddf74 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:22.428719 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709850ac-362d-4e62-8b18-4897d24ddf74 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:22.429177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709850ac-362d-4e62-8b18-4897d24ddf74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:22.443309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-39ae1642-c578-4a87-8f9a-93ae81c8947c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a4ddecea1cd4714a724ffe5c1231e04', 'request_id': 'req-39ae1642-c578-4a87-8f9a-93ae81c8947c'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:22.456829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-39ae1642-c578-4a87-8f9a-93ae81c8947c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a4ddecea1cd4714a724ffe5c1231e04', 'request_id': 'req-39ae1642-c578-4a87-8f9a-93ae81c8947c'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:22.459353 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:22.470394 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-709850ac-362d-4e62-8b18-4897d24ddf74 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:22.472672 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-709850ac-362d-4e62-8b18-4897d24ddf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:22.501743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-39ae1642-c578-4a87-8f9a-93ae81c8947c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a4ddecea1cd4714a724ffe5c1231e04', 'request_id': 'req-39ae1642-c578-4a87-8f9a-93ae81c8947c'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:22.558728 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-39ae1642-c578-4a87-8f9a-93ae81c8947c admin admin] The token cache is being invalidate because project 2a4ddecea1cd4714a724ffe5c1231e04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:34:22.559134 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-39ae1642-c578-4a87-8f9a-93ae81c8947c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:22.562507 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-39ae1642-c578-4a87-8f9a-93ae81c8947c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:22.567456 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 841/1676] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:34:22 2022] DELETE /identity/v3/projects/2a4ddecea1cd4714a724ffe5c1231e04 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:34:22.571256 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae1642-c578-4a87-8f9a-93ae81c8947c admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:22.576571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-39ae1642-c578-4a87-8f9a-93ae81c8947c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:22.630967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 836/1677] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:22.638958 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea49bc-1e4c-4c4a-bd94-25cadebcb574 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:22.639128 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea49bc-1e4c-4c4a-bd94-25cadebcb574 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:22.639566 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea49bc-1e4c-4c4a-bd94-25cadebcb574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:22.701473 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d2ea49bc-1e4c-4c4a-bd94-25cadebcb574 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:22.705105 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae1642-c578-4a87-8f9a-93ae81c8947c admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:22.705486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d2ea49bc-1e4c-4c4a-bd94-25cadebcb574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:22.716794 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467cf97-6b52-41f0-a4f6-96208be63581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:22.717761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467cf97-6b52-41f0-a4f6-96208be63581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:22.718049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467cf97-6b52-41f0-a4f6-96208be63581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:22.719941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467cf97-6b52-41f0-a4f6-96208be63581 admin admin] PATH_INFO: `/v3/users/1016e143d9f4473c9ce7b5c1c01ae743` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:22.721967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9467cf97-6b52-41f0-a4f6-96208be63581 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1016e143d9f4473c9ce7b5c1c01ae743)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:22.727100 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 837/1678] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:22.728287 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea49bc-1e4c-4c4a-bd94-25cadebcb574 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:22.729845 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9467cf97-6b52-41f0-a4f6-96208be63581 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:22.766869 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea49bc-1e4c-4c4a-bd94-25cadebcb574 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:22.775609 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626fd8a0-b43d-408f-8065-39e4c41a73c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:22.776608 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fd8a0-b43d-408f-8065-39e4c41a73c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:22.777244 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fd8a0-b43d-408f-8065-39e4c41a73c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:22.777886 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fd8a0-b43d-408f-8065-39e4c41a73c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:22.779023 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626fd8a0-b43d-408f-8065-39e4c41a73c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:22.779607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626fd8a0-b43d-408f-8065-39e4c41a73c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:22.782290 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626fd8a0-b43d-408f-8065-39e4c41a73c8 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:22.807882 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 838/1679] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:22.809431 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626fd8a0-b43d-408f-8065-39e4c41a73c8 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:22.823678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626fd8a0-b43d-408f-8065-39e4c41a73c8 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:22.837722 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4195b9b-e54d-455f-a732-b91a63c8d4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:22.838966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4195b9b-e54d-455f-a732-b91a63c8d4e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:22.839482 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4195b9b-e54d-455f-a732-b91a63c8d4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:22.839983 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4195b9b-e54d-455f-a732-b91a63c8d4e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:22.841075 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4195b9b-e54d-455f-a732-b91a63c8d4e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:22.845476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4195b9b-e54d-455f-a732-b91a63c8d4e3 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:22.851701 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9467cf97-6b52-41f0-a4f6-96208be63581 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1016e143d9f4473c9ce7b5c1c01ae743'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:22.869925 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9467cf97-6b52-41f0-a4f6-96208be63581 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1016e143d9f4473c9ce7b5c1c01ae743'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:22.872198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9467cf97-6b52-41f0-a4f6-96208be63581 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1016e143d9f4473c9ce7b5c1c01ae743'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:22.890243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9467cf97-6b52-41f0-a4f6-96208be63581 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1016e143d9f4473c9ce7b5c1c01ae743'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:22.920490 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9467cf97-6b52-41f0-a4f6-96208be63581 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1016e143d9f4473c9ce7b5c1c01ae743'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:22.927585 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 839/1680] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:22 2022] POST /identity/v3/projects => generated 357 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:22.933752 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4195b9b-e54d-455f-a732-b91a63c8d4e3 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:22.938682 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a4195b9b-e54d-455f-a732-b91a63c8d4e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:22.942624 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 842/1681] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:34:22 2022] DELETE /identity/v3/users/1016e143d9f4473c9ce7b5c1c01ae743 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:22.997226 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467cf97-6b52-41f0-a4f6-96208be63581 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:22.999795 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-9467cf97-6b52-41f0-a4f6-96208be63581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:23.090747 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:23.151122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4195b9b-e54d-455f-a732-b91a63c8d4e3 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:23.188777 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467cf97-6b52-41f0-a4f6-96208be63581 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:23.196411 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec3aa4-a66d-4708-b514-8bb469b75282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:23.198244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec3aa4-a66d-4708-b514-8bb469b75282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:23.198244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec3aa4-a66d-4708-b514-8bb469b75282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:23.198711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec3aa4-a66d-4708-b514-8bb469b75282 admin admin] PATH_INFO: `/v3/users/fa4dfe8c58e8489ebcf5c6641bd34c2a` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:23.202628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec3aa4-a66d-4708-b514-8bb469b75282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa4dfe8c58e8489ebcf5c6641bd34c2a)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:23.207657 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec3aa4-a66d-4708-b514-8bb469b75282 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:23.219667 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:23.322948 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-55ec3aa4-a66d-4708-b514-8bb469b75282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa4dfe8c58e8489ebcf5c6641bd34c2a'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:23.336698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbcfec5-743f-4028-83ec-335da2dcc6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:23.336967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-55ec3aa4-a66d-4708-b514-8bb469b75282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa4dfe8c58e8489ebcf5c6641bd34c2a'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:23.337917 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-55ec3aa4-a66d-4708-b514-8bb469b75282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa4dfe8c58e8489ebcf5c6641bd34c2a'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:23.338098 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbcfec5-743f-4028-83ec-335da2dcc6f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:23.338493 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbcfec5-743f-4028-83ec-335da2dcc6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:23.338883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbcfec5-743f-4028-83ec-335da2dcc6f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:23.339740 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbcfec5-743f-4028-83ec-335da2dcc6f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:23.343827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbcfec5-743f-4028-83ec-335da2dcc6f3 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:23.350402 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-55ec3aa4-a66d-4708-b514-8bb469b75282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa4dfe8c58e8489ebcf5c6641bd34c2a'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:23.367679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-55ec3aa4-a66d-4708-b514-8bb469b75282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa4dfe8c58e8489ebcf5c6641bd34c2a'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:23.382186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 843/1682] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:34:22 2022] DELETE /identity/v3/users/fa4dfe8c58e8489ebcf5c6641bd34c2a => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:23.383954 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6325a-42bf-4b3a-a705-d75b637fa708 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:23.384301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6325a-42bf-4b3a-a705-d75b637fa708 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:23.384856 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6325a-42bf-4b3a-a705-d75b637fa708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:23.403992 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 840/1683] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:22 2022] POST /identity/v3/users => generated 279 bytes in 470 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:23.406441 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50736ba6-c677-427d-bf06-5ad6e218e59b None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:23.408151 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50736ba6-c677-427d-bf06-5ad6e218e59b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:23.408542 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50736ba6-c677-427d-bf06-5ad6e218e59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:23.450306 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-50736ba6-c677-427d-bf06-5ad6e218e59b None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:23.450554 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-83b6325a-42bf-4b3a-a705-d75b637fa708 None None] MFA Rules not processed for user `a5c37b79bf3d40f4a6182cc296585c18`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:23.451785 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-50736ba6-c677-427d-bf06-5ad6e218e59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:23.451896 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-83b6325a-42bf-4b3a-a705-d75b637fa708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:23.597664 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 841/1684] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:23.598466 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50736ba6-c677-427d-bf06-5ad6e218e59b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:23.602440 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-50736ba6-c677-427d-bf06-5ad6e218e59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:23.624505 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50736ba6-c677-427d-bf06-5ad6e218e59b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:23.645875 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b025da-e2b3-49d4-8691-ce4f3605f5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:23.646759 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b025da-e2b3-49d4-8691-ce4f3605f5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:23.647153 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b025da-e2b3-49d4-8691-ce4f3605f5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:23.647562 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b025da-e2b3-49d4-8691-ce4f3605f5e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:23.648834 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b025da-e2b3-49d4-8691-ce4f3605f5e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:23.651118 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b025da-e2b3-49d4-8691-ce4f3605f5e6 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:23.680635 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 842/1685] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:23.684953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b025da-e2b3-49d4-8691-ce4f3605f5e6 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:23.691818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f3b025da-e2b3-49d4-8691-ce4f3605f5e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:23.724305 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 844/1686] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:23.725438 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6325a-42bf-4b3a-a705-d75b637fa708 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:23.743371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6325a-42bf-4b3a-a705-d75b637fa708 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:23.750500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce8e2d-7f25-47e7-8ff0-7d4198f42851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:23.751548 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce8e2d-7f25-47e7-8ff0-7d4198f42851 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:23.751925 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce8e2d-7f25-47e7-8ff0-7d4198f42851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:23.752267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce8e2d-7f25-47e7-8ff0-7d4198f42851 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:23.755351 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce8e2d-7f25-47e7-8ff0-7d4198f42851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:23.757851 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce8e2d-7f25-47e7-8ff0-7d4198f42851 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:23.787954 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 845/1687] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:23.789550 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dba58-64f6-4460-be16-b6742651f735 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:23.789913 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dba58-64f6-4460-be16-b6742651f735 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:23.790208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dba58-64f6-4460-be16-b6742651f735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:23.814523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-ea1dba58-64f6-4460-be16-b6742651f735 None None] MFA Rules not processed for user `a5c37b79bf3d40f4a6182cc296585c18`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:23.816610 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-ea1dba58-64f6-4460-be16-b6742651f735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:23.833608 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 846/1688] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:23.834570 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dba58-64f6-4460-be16-b6742651f735 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:23.847339 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dba58-64f6-4460-be16-b6742651f735 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:23.854730 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0b67b-a32c-4aa8-a59e-22fa3eba1800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:23.855661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0b67b-a32c-4aa8-a59e-22fa3eba1800 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:23.856094 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0b67b-a32c-4aa8-a59e-22fa3eba1800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:23.856482 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0b67b-a32c-4aa8-a59e-22fa3eba1800 admin admin] PATH_INFO: `/v3/projects/3055a17e0b6c4d579559f55e949a98a9/users/c3864bcfbb384b8c8dbdf8bb63b8074a/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:23.874623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0b67b-a32c-4aa8-a59e-22fa3eba1800 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3055a17e0b6c4d579559f55e949a98a9, user_id=c3864bcfbb384b8c8dbdf8bb63b8074a, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:23.877587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0b67b-a32c-4aa8-a59e-22fa3eba1800 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:23.885176 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b025da-e2b3-49d4-8691-ce4f3605f5e6 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:23.896106 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 847/1689] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:23 2022] PUT /identity/v3/projects/3055a17e0b6c4d579559f55e949a98a9/users/c3864bcfbb384b8c8dbdf8bb63b8074a/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:23.897406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0b67b-a32c-4aa8-a59e-22fa3eba1800 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:23.899635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-32d0b67b-a32c-4aa8-a59e-22fa3eba1800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:23.949175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:24.003554 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:24.036402 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ecf8e-2d86-4ff9-9be5-4efefb555546 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:24.039902 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ecf8e-2d86-4ff9-9be5-4efefb555546 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:24.040960 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ecf8e-2d86-4ff9-9be5-4efefb555546 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:24.040960 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ecf8e-2d86-4ff9-9be5-4efefb555546 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:24.096232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0b67b-a32c-4aa8-a59e-22fa3eba1800 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:24.111321 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856324a1-7d0d-4db2-8ce0-5c62156c0ec9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:24.113056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856324a1-7d0d-4db2-8ce0-5c62156c0ec9 service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:24.113056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856324a1-7d0d-4db2-8ce0-5c62156c0ec9 service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:24.113378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856324a1-7d0d-4db2-8ce0-5c62156c0ec9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:24.172101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ecf8e-2d86-4ff9-9be5-4efefb555546 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:24.176681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ecf8e-2d86-4ff9-9be5-4efefb555546 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:24.190175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 843/1690] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:24.193625 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3b3bc-16d8-466b-988c-80cbf5d2db6b None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:24.194107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3b3bc-16d8-466b-988c-80cbf5d2db6b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:24.194532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3b3bc-16d8-466b-988c-80cbf5d2db6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:24.215749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856324a1-7d0d-4db2-8ce0-5c62156c0ec9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:24.217406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856324a1-7d0d-4db2-8ce0-5c62156c0ec9 service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:24.229518 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 848/1691] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:24.254066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-efc3b3bc-16d8-466b-988c-80cbf5d2db6b None None] MFA Rules not processed for user `c3864bcfbb384b8c8dbdf8bb63b8074a`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:24.256759 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-efc3b3bc-16d8-466b-988c-80cbf5d2db6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:24.411976 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 844/1692] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:24 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:24.416808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856324a1-7d0d-4db2-8ce0-5c62156c0ec9 service nova] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:24.477999 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856324a1-7d0d-4db2-8ce0-5c62156c0ec9 service nova] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:24.498409 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb202ff7-6429-43cd-a70b-432ef65959d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:24.499141 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb202ff7-6429-43cd-a70b-432ef65959d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:24.499442 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb202ff7-6429-43cd-a70b-432ef65959d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:24.499734 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb202ff7-6429-43cd-a70b-432ef65959d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:24.508877 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb202ff7-6429-43cd-a70b-432ef65959d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:24.524276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a486f-fdb9-468b-98ea-779969ec6117 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:24.524426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb202ff7-6429-43cd-a70b-432ef65959d7 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:24.524887 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a486f-fdb9-468b-98ea-779969ec6117 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:24.525369 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a486f-fdb9-468b-98ea-779969ec6117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:24.576344 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 849/1693] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:24 2022] POST /identity/v3/projects => generated 355 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:24.580873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb202ff7-6429-43cd-a70b-432ef65959d7 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:24.594048 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-981a486f-fdb9-468b-98ea-779969ec6117 None None] MFA Rules not processed for user `cbe7720ab8b7414a8c336de368ae1ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:24.595879 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-981a486f-fdb9-468b-98ea-779969ec6117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:24.686957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb202ff7-6429-43cd-a70b-432ef65959d7 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:24.695101 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b06240-ed81-49b4-a8b4-6ebff231cee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:24.695811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b06240-ed81-49b4-a8b4-6ebff231cee9 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:24.696107 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b06240-ed81-49b4-a8b4-6ebff231cee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:24.696394 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b06240-ed81-49b4-a8b4-6ebff231cee9 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:24.697403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b06240-ed81-49b4-a8b4-6ebff231cee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:24.699603 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b06240-ed81-49b4-a8b4-6ebff231cee9 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:24.776451 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 850/1694] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:24 2022] POST /identity/v3/users => generated 284 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:24.780214 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b06240-ed81-49b4-a8b4-6ebff231cee9 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:24.785659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:24.814187 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b06240-ed81-49b4-a8b4-6ebff231cee9 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:24.824909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af20ad2-ca2a-4202-877f-e1863780fd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:24.826079 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af20ad2-ca2a-4202-877f-e1863780fd6d admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:24.826594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af20ad2-ca2a-4202-877f-e1863780fd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:24.827221 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af20ad2-ca2a-4202-877f-e1863780fd6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:24.828231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af20ad2-ca2a-4202-877f-e1863780fd6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:24.829782 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 845/1695] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:24.830789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af20ad2-ca2a-4202-877f-e1863780fd6d admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:24.847806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e15bc-6956-4392-867e-7bfa69b07b4d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:24.848206 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e15bc-6956-4392-867e-7bfa69b07b4d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:24.848206 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e15bc-6956-4392-867e-7bfa69b07b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:24.854505 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 851/1696] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:24.858669 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af20ad2-ca2a-4202-877f-e1863780fd6d admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:24.873992 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af20ad2-ca2a-4202-877f-e1863780fd6d admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:24.881453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aee955-edae-48bc-a11b-5b38b2c55d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:24.882719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aee955-edae-48bc-a11b-5b38b2c55d2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:24.883217 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aee955-edae-48bc-a11b-5b38b2c55d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:24.883702 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aee955-edae-48bc-a11b-5b38b2c55d2b admin admin] PATH_INFO: `/v3/projects/0ba32a55e31d4a3b972c75e35236ce99/users/a6f128138b9a4a748ba7203581a7e6c1/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:24.903040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1e4e15bc-6956-4392-867e-7bfa69b07b4d None None] MFA Rules not processed for user `cbe7720ab8b7414a8c336de368ae1ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:24.906889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aee955-edae-48bc-a11b-5b38b2c55d2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba32a55e31d4a3b972c75e35236ce99, user_id=a6f128138b9a4a748ba7203581a7e6c1, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:24.911119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aee955-edae-48bc-a11b-5b38b2c55d2b admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:24.915101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1e4e15bc-6956-4392-867e-7bfa69b07b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:24.942637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 846/1697] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:24.954336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e15bc-6956-4392-867e-7bfa69b07b4d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:24.959184 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1e4e15bc-6956-4392-867e-7bfa69b07b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:24.970471 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 852/1698] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:24 2022] PUT /identity/v3/projects/0ba32a55e31d4a3b972c75e35236ce99/users/a6f128138b9a4a748ba7203581a7e6c1/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:24.975163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aee955-edae-48bc-a11b-5b38b2c55d2b admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:24.985034 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:25.154407 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aee955-edae-48bc-a11b-5b38b2c55d2b admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:25.169423 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89fdba-a3a6-4683-9904-c5fe80a68ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:25.171084 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89fdba-a3a6-4683-9904-c5fe80a68ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:25.171518 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89fdba-a3a6-4683-9904-c5fe80a68ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:25.171823 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89fdba-a3a6-4683-9904-c5fe80a68ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:25.174734 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89fdba-a3a6-4683-9904-c5fe80a68ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:25.179391 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89fdba-a3a6-4683-9904-c5fe80a68ab2 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:25.204464 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 853/1699] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:34:25.209595 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89fdba-a3a6-4683-9904-c5fe80a68ab2 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:25.229920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e15bc-6956-4392-867e-7bfa69b07b4d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:25.233538 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89fdba-a3a6-4683-9904-c5fe80a68ab2 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:25.255675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc99cf-145b-46d1-b45c-d01baa5efb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:25.257166 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc99cf-145b-46d1-b45c-d01baa5efb0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:25.257166 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc99cf-145b-46d1-b45c-d01baa5efb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:25.257442 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc99cf-145b-46d1-b45c-d01baa5efb0e admin admin] PATH_INFO: `/v3/projects/0ba32a55e31d4a3b972c75e35236ce99/users/a6f128138b9a4a748ba7203581a7e6c1/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:25.259839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc99cf-145b-46d1-b45c-d01baa5efb0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba32a55e31d4a3b972c75e35236ce99, user_id=a6f128138b9a4a748ba7203581a7e6c1, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:25.263297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc99cf-145b-46d1-b45c-d01baa5efb0e admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:25.291923 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 854/1700] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:25 2022] PUT /identity/v3/projects/0ba32a55e31d4a3b972c75e35236ce99/users/a6f128138b9a4a748ba7203581a7e6c1/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:25.306512 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7cf09-ee0f-453a-8482-500d7762ac1b None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:25.306512 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7cf09-ee0f-453a-8482-500d7762ac1b None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:25.306754 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7cf09-ee0f-453a-8482-500d7762ac1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:25.362106 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-dbe7cf09-ee0f-453a-8482-500d7762ac1b None None] MFA Rules not processed for user `a6f128138b9a4a748ba7203581a7e6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:25.367518 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-dbe7cf09-ee0f-453a-8482-500d7762ac1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:25.502935 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8870ee-0f8f-420b-96c7-cac8f072082c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:25.504869 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8870ee-0f8f-420b-96c7-cac8f072082c service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:25.505176 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8870ee-0f8f-420b-96c7-cac8f072082c service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:25.507481 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8870ee-0f8f-420b-96c7-cac8f072082c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:25.545089 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 855/1701] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:25 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:34:25.560651 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7cf09-ee0f-453a-8482-500d7762ac1b None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:25.562972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-dbe7cf09-ee0f-453a-8482-500d7762ac1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:25.644720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8870ee-0f8f-420b-96c7-cac8f072082c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:25.647053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8870ee-0f8f-420b-96c7-cac8f072082c service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:25.683427 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 847/1702] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 729 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:25.698739 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7cf09-ee0f-453a-8482-500d7762ac1b None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:25.707987 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3ea0e-ab2b-4236-8900-9dccb223f0c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:25.712878 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3ea0e-ab2b-4236-8900-9dccb223f0c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:25.713214 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3ea0e-ab2b-4236-8900-9dccb223f0c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:25.713585 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3ea0e-ab2b-4236-8900-9dccb223f0c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:25.738429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3ea0e-ab2b-4236-8900-9dccb223f0c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:25.740429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3ea0e-ab2b-4236-8900-9dccb223f0c0 service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:25.787021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 856/1703] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:25 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:34:25.840237 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cf747-def0-4295-bfb4-97e81d1898e1 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:25.840969 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cf747-def0-4295-bfb4-97e81d1898e1 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:25.844354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cf747-def0-4295-bfb4-97e81d1898e1 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:25.844354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 848/1704] 10.222.0.40 () {54 vars in 912 bytes} [Thu Jun 16 17:34:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:34:25.856338 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a612e-934a-487e-b769-22a5e1bbd8c8 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:25.856906 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a612e-934a-487e-b769-22a5e1bbd8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:25.857565 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a612e-934a-487e-b769-22a5e1bbd8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:25.883295 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cf747-def0-4295-bfb4-97e81d1898e1 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:25.885039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d07cf747-def0-4295-bfb4-97e81d1898e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:25.926659 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:25.972263 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-693a612e-934a-487e-b769-22a5e1bbd8c8 None None] MFA Rules not processed for user `a303489493ce4cd2b62e876fbc8a7404`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:25.980714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-693a612e-934a-487e-b769-22a5e1bbd8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:25.993532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:26.116126 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cf747-def0-4295-bfb4-97e81d1898e1 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:26.132018 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568aecf-2921-4939-ac69-ac58873e1521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:26.132967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568aecf-2921-4939-ac69-ac58873e1521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:26.135942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568aecf-2921-4939-ac69-ac58873e1521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:26.135942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568aecf-2921-4939-ac69-ac58873e1521 admin admin] PATH_INFO: `/v3/projects/bde6161bc2d74d24b4a4ab0c0d4de7ba` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:26.138686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568aecf-2921-4939-ac69-ac58873e1521 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bde6161bc2d74d24b4a4ab0c0d4de7ba)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:26.146627 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568aecf-2921-4939-ac69-ac58873e1521 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:26.154132 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 857/1705] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 17:34:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:26.157294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a612e-934a-487e-b769-22a5e1bbd8c8 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:26.178370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a612e-934a-487e-b769-22a5e1bbd8c8 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:26.186401 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51c323-8ab2-4bb6-9958-f9e5f8350575 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:26.187075 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51c323-8ab2-4bb6-9958-f9e5f8350575 service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:26.187363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51c323-8ab2-4bb6-9958-f9e5f8350575 service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:26.187683 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51c323-8ab2-4bb6-9958-f9e5f8350575 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:26.274572 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c568aecf-2921-4939-ac69-ac58873e1521 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bde6161bc2d74d24b4a4ab0c0d4de7ba', 'request_id': 'req-c568aecf-2921-4939-ac69-ac58873e1521'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:26.301954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c568aecf-2921-4939-ac69-ac58873e1521 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bde6161bc2d74d24b4a4ab0c0d4de7ba', 'request_id': 'req-c568aecf-2921-4939-ac69-ac58873e1521'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:26.313237 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c568aecf-2921-4939-ac69-ac58873e1521 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bde6161bc2d74d24b4a4ab0c0d4de7ba', 'request_id': 'req-c568aecf-2921-4939-ac69-ac58873e1521'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:26.339655 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51c323-8ab2-4bb6-9958-f9e5f8350575 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:26.350792 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51c323-8ab2-4bb6-9958-f9e5f8350575 service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:26.358724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c568aecf-2921-4939-ac69-ac58873e1521 admin admin] The token cache is being invalidate because project bde6161bc2d74d24b4a4ab0c0d4de7ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:34:26.359296 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c568aecf-2921-4939-ac69-ac58873e1521 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:26.361163 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c568aecf-2921-4939-ac69-ac58873e1521 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:26.369308 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 849/1706] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:34:25 2022] DELETE /identity/v3/projects/bde6161bc2d74d24b4a4ab0c0d4de7ba => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:34:26.370455 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568aecf-2921-4939-ac69-ac58873e1521 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:26.372172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c568aecf-2921-4939-ac69-ac58873e1521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:26.402399 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 858/1707] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:26 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:26.408069 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51c323-8ab2-4bb6-9958-f9e5f8350575 service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:26.443735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51c323-8ab2-4bb6-9958-f9e5f8350575 service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:26.585380 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a227d71-dbd3-4905-b143-553bd6efb60e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:26.586303 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a227d71-dbd3-4905-b143-553bd6efb60e service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:26.586740 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a227d71-dbd3-4905-b143-553bd6efb60e service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:26.587556 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a227d71-dbd3-4905-b143-553bd6efb60e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:26.589721 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-5a227d71-dbd3-4905-b143-553bd6efb60e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:26.615836 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a227d71-dbd3-4905-b143-553bd6efb60e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:26.616980 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568aecf-2921-4939-ac69-ac58873e1521 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:26.630074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5f662-a832-438d-b415-fb245fc58f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:26.631276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5f662-a832-438d-b415-fb245fc58f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:26.631735 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5f662-a832-438d-b415-fb245fc58f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:26.632256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5f662-a832-438d-b415-fb245fc58f94 admin admin] PATH_INFO: `/v3/projects/565a11ea3a9f475089782a186af4991d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:26.633813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a227d71-dbd3-4905-b143-553bd6efb60e service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:26.634238 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5f662-a832-438d-b415-fb245fc58f94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=565a11ea3a9f475089782a186af4991d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:26.636607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5f662-a832-438d-b415-fb245fc58f94 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:26.668769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 859/1708] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:26.670371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b7fa7-c3bc-471e-8ab5-84efcccfa76c None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:26.670797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b7fa7-c3bc-471e-8ab5-84efcccfa76c None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:26.670992 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b7fa7-c3bc-471e-8ab5-84efcccfa76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:26.716553 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-737b7fa7-c3bc-471e-8ab5-84efcccfa76c None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:26.722611 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-737b7fa7-c3bc-471e-8ab5-84efcccfa76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:26.741524 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 860/1709] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:26.744401 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef47df3-0922-4695-8472-dd561f033fb2 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:26.744707 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef47df3-0922-4695-8472-dd561f033fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:26.744997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef47df3-0922-4695-8472-dd561f033fb2 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:26.756592 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 861/1710] 10.222.0.40 () {54 vars in 912 bytes} [Thu Jun 16 17:34:26 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:34:26.758737 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb61ca7-3001-4a5b-9f4b-cafc8f29d322 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:26.759031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb61ca7-3001-4a5b-9f4b-cafc8f29d322 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:26.760227 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb61ca7-3001-4a5b-9f4b-cafc8f29d322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:26.798476 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-fbb61ca7-3001-4a5b-9f4b-cafc8f29d322 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:26.800838 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-fbb61ca7-3001-4a5b-9f4b-cafc8f29d322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:26.853942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cca5f662-a832-438d-b415-fb245fc58f94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '565a11ea3a9f475089782a186af4991d', 'request_id': 'req-cca5f662-a832-438d-b415-fb245fc58f94'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:26.860029 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:26.869724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cca5f662-a832-438d-b415-fb245fc58f94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '565a11ea3a9f475089782a186af4991d', 'request_id': 'req-cca5f662-a832-438d-b415-fb245fc58f94'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:26.882234 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cca5f662-a832-438d-b415-fb245fc58f94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '565a11ea3a9f475089782a186af4991d', 'request_id': 'req-cca5f662-a832-438d-b415-fb245fc58f94'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:26.908657 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cca5f662-a832-438d-b415-fb245fc58f94 admin admin] The token cache is being invalidate because project 565a11ea3a9f475089782a186af4991d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:34:26.910249 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cca5f662-a832-438d-b415-fb245fc58f94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:26.911334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cca5f662-a832-438d-b415-fb245fc58f94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:26.913856 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 850/1711] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:34:26 2022] DELETE /identity/v3/projects/565a11ea3a9f475089782a186af4991d => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:26.916027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79935fda-44e6-47de-bb52-bccf77166bf5 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:26.916456 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79935fda-44e6-47de-bb52-bccf77166bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:26.917728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79935fda-44e6-47de-bb52-bccf77166bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:26.948106 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:26.972608 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-79935fda-44e6-47de-bb52-bccf77166bf5 None None] MFA Rules not processed for user `a303489493ce4cd2b62e876fbc8a7404`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:26.976374 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-79935fda-44e6-47de-bb52-bccf77166bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:27.026394 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 862/1712] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:27.032537 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b31b8-be8e-430d-af95-a405310509aa None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:27.032842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b31b8-be8e-430d-af95-a405310509aa None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:27.033137 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b31b8-be8e-430d-af95-a405310509aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:27.096541 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-d68b31b8-be8e-430d-af95-a405310509aa None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:27.102935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-d68b31b8-be8e-430d-af95-a405310509aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:27.140676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 863/1713] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:27.141676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b31b8-be8e-430d-af95-a405310509aa None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:27.143968 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 851/1714] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 17:34:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:27.150332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b230a-038c-40ab-955d-83e2a9c1c91f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:27.150709 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b230a-038c-40ab-955d-83e2a9c1c91f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:27.151618 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b230a-038c-40ab-955d-83e2a9c1c91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:27.176361 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b31b8-be8e-430d-af95-a405310509aa None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:27.197232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f77232-9818-4fb3-9bed-40b3036c240c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:27.198001 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f77232-9818-4fb3-9bed-40b3036c240c admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:27.198292 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f77232-9818-4fb3-9bed-40b3036c240c admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:27.198577 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f77232-9818-4fb3-9bed-40b3036c240c admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:27.199348 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f77232-9818-4fb3-9bed-40b3036c240c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:27.201872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f77232-9818-4fb3-9bed-40b3036c240c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:27.203160 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-695b230a-038c-40ab-955d-83e2a9c1c91f None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:27.205431 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-695b230a-038c-40ab-955d-83e2a9c1c91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:27.210640 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f77232-9818-4fb3-9bed-40b3036c240c admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:27.244059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 852/1715] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:27.244059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b230a-038c-40ab-955d-83e2a9c1c91f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:27.248592 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-695b230a-038c-40ab-955d-83e2a9c1c91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:27.253731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 864/1716] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:27.254314 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f77232-9818-4fb3-9bed-40b3036c240c admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:27.279110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b230a-038c-40ab-955d-83e2a9c1c91f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:27.288090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f77232-9818-4fb3-9bed-40b3036c240c admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:27.289514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac612553-5d35-4eb4-863e-b618b032bdff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:27.290424 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac612553-5d35-4eb4-863e-b618b032bdff service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:27.293239 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac612553-5d35-4eb4-863e-b618b032bdff service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:27.293814 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac612553-5d35-4eb4-863e-b618b032bdff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:27.313759 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84b618-3a72-4a35-b2ba-a17aef3a670a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:27.314462 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84b618-3a72-4a35-b2ba-a17aef3a670a admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:27.314758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84b618-3a72-4a35-b2ba-a17aef3a670a admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:27.315053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84b618-3a72-4a35-b2ba-a17aef3a670a admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:27.317304 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84b618-3a72-4a35-b2ba-a17aef3a670a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:27.317304 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84b618-3a72-4a35-b2ba-a17aef3a670a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:27.317648 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac612553-5d35-4eb4-863e-b618b032bdff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:27.322341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84b618-3a72-4a35-b2ba-a17aef3a670a admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:27.334779 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac612553-5d35-4eb4-863e-b618b032bdff service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:27.351836 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 865/1717] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:27.356408 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84b618-3a72-4a35-b2ba-a17aef3a670a admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:27.378897 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 853/1718] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:27.379898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac612553-5d35-4eb4-863e-b618b032bdff service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:27.398094 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84b618-3a72-4a35-b2ba-a17aef3a670a admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:27.421982 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac612553-5d35-4eb4-863e-b618b032bdff service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:27.423559 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cbff3-778c-4318-b133-a3c27c9f1959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:27.424275 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cbff3-778c-4318-b133-a3c27c9f1959 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:27.424568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cbff3-778c-4318-b133-a3c27c9f1959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:27.428792 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cbff3-778c-4318-b133-a3c27c9f1959 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:27.430535 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cbff3-778c-4318-b133-a3c27c9f1959 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:27.432544 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cbff3-778c-4318-b133-a3c27c9f1959 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:27.452924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bd0de-64c6-42c0-b126-c1e9a2778505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:27.452924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bd0de-64c6-42c0-b126-c1e9a2778505 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:27.452924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bd0de-64c6-42c0-b126-c1e9a2778505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:27.453275 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bd0de-64c6-42c0-b126-c1e9a2778505 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:27.461983 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1bd0de-64c6-42c0-b126-c1e9a2778505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:27.468473 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1bd0de-64c6-42c0-b126-c1e9a2778505 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:27.515368 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 854/1719] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:27 2022] POST /identity/v3/projects => generated 379 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:27.521003 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bd0de-64c6-42c0-b126-c1e9a2778505 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:27.533774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 866/1720] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:27 2022] POST /identity/v3/projects => generated 401 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:27.537894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cbff3-778c-4318-b133-a3c27c9f1959 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:27.686680 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bd0de-64c6-42c0-b126-c1e9a2778505 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:27.759190 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cbff3-778c-4318-b133-a3c27c9f1959 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:27.777743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3108086-f397-4308-9d29-b4624304f78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:27.778455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3108086-f397-4308-9d29-b4624304f78d admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:27.778748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3108086-f397-4308-9d29-b4624304f78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:27.779031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3108086-f397-4308-9d29-b4624304f78d admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:27.783029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3108086-f397-4308-9d29-b4624304f78d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:27.790695 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3108086-f397-4308-9d29-b4624304f78d admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:27.841914 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 867/1721] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:27 2022] POST /identity/v3/users => generated 301 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:27.843053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3108086-f397-4308-9d29-b4624304f78d admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:27.868464 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:27.872135 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3108086-f397-4308-9d29-b4624304f78d admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:27.888426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1004d1c-3322-4342-aec1-65ad4b068695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:27.891684 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a389c6-939c-4c43-8c35-5ecc9f965630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:27.892641 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1004d1c-3322-4342-aec1-65ad4b068695 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:27.892942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a389c6-939c-4c43-8c35-5ecc9f965630 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:27.893255 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a389c6-939c-4c43-8c35-5ecc9f965630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:27.893569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a389c6-939c-4c43-8c35-5ecc9f965630 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:27.894305 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a389c6-939c-4c43-8c35-5ecc9f965630 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:27.894890 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1004d1c-3322-4342-aec1-65ad4b068695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:27.895351 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1004d1c-3322-4342-aec1-65ad4b068695 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:27.897432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1004d1c-3322-4342-aec1-65ad4b068695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:27.903313 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1004d1c-3322-4342-aec1-65ad4b068695 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:27.905698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a389c6-939c-4c43-8c35-5ecc9f965630 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:27.954636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 868/1722] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:34:27.960197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1004d1c-3322-4342-aec1-65ad4b068695 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:27.974091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1004d1c-3322-4342-aec1-65ad4b068695 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:27.980351 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9b916-ce64-4e30-8711-97f48cf3f69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:27.981177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9b916-ce64-4e30-8711-97f48cf3f69b admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:27.981528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9b916-ce64-4e30-8711-97f48cf3f69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:27.982005 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9b916-ce64-4e30-8711-97f48cf3f69b admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:27.982936 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9b916-ce64-4e30-8711-97f48cf3f69b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:27.984640 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9b916-ce64-4e30-8711-97f48cf3f69b admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:27.997437 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 855/1723] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:27 2022] POST /identity/v3/users => generated 290 bytes in 477 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:28.006405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 869/1724] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:28.007882 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9b916-ce64-4e30-8711-97f48cf3f69b admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:28.008213 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a389c6-939c-4c43-8c35-5ecc9f965630 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:28.038224 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a389c6-939c-4c43-8c35-5ecc9f965630 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:28.053952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dc016-b48c-44f8-848f-1159b9cbce62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:28.056880 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9b916-ce64-4e30-8711-97f48cf3f69b admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:28.068498 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dc016-b48c-44f8-848f-1159b9cbce62 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:28.068779 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dc016-b48c-44f8-848f-1159b9cbce62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:28.069067 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dc016-b48c-44f8-848f-1159b9cbce62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:28.069858 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647dc016-b48c-44f8-848f-1159b9cbce62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:28.071720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647dc016-b48c-44f8-848f-1159b9cbce62 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:28.079655 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca9b92-2cef-422d-ae17-7fd033801159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:28.080611 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca9b92-2cef-422d-ae17-7fd033801159 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:28.081600 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca9b92-2cef-422d-ae17-7fd033801159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:28.081853 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca9b92-2cef-422d-ae17-7fd033801159 admin admin] PATH_INFO: `/v3/projects/8ae33446bcab410695163c6bb2be7afe/users/3583c5993d4c4b3fab50eed455995fc2/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:28.118705 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 856/1725] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:28.119918 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee00b1-7139-4623-8727-396b1e8fa3bc None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:28.120215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee00b1-7139-4623-8727-396b1e8fa3bc None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:28.120504 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee00b1-7139-4623-8727-396b1e8fa3bc None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:28.129869 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 857/1726] 10.222.0.40 () {54 vars in 960 bytes} [Thu Jun 16 17:34:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:34:28.130191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee00b1-7139-4623-8727-396b1e8fa3bc None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:28.151463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca9b92-2cef-422d-ae17-7fd033801159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae33446bcab410695163c6bb2be7afe, user_id=3583c5993d4c4b3fab50eed455995fc2, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:28.156750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca9b92-2cef-422d-ae17-7fd033801159 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:28.163940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee00b1-7139-4623-8727-396b1e8fa3bc None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:28.175093 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86664fdb-ff07-4e66-9cdf-db9a0e173304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:28.175807 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86664fdb-ff07-4e66-9cdf-db9a0e173304 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:28.176166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86664fdb-ff07-4e66-9cdf-db9a0e173304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:28.176543 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86664fdb-ff07-4e66-9cdf-db9a0e173304 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:28.177321 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86664fdb-ff07-4e66-9cdf-db9a0e173304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:28.179064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86664fdb-ff07-4e66-9cdf-db9a0e173304 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:28.184206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 870/1727] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:28 2022] PUT /identity/v3/projects/8ae33446bcab410695163c6bb2be7afe/users/3583c5993d4c4b3fab50eed455995fc2/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:28.186403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdb80c-8ee7-4070-97b2-24eadebaf3e1 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:28.186867 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdb80c-8ee7-4070-97b2-24eadebaf3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:28.186981 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdb80c-8ee7-4070-97b2-24eadebaf3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:28.223234 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-6ecdb80c-8ee7-4070-97b2-24eadebaf3e1 None None] MFA Rules not processed for user `75f0586a8c1941448d750918183f6dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:28.226281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 858/1728] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:28.233782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-6ecdb80c-8ee7-4070-97b2-24eadebaf3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:28.252677 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb326a9-fb74-4701-9350-2754e780f5bf None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:28.252677 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb326a9-fb74-4701-9350-2754e780f5bf None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:28.253371 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb326a9-fb74-4701-9350-2754e780f5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:28.310833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-fcb326a9-fb74-4701-9350-2754e780f5bf None None] MFA Rules not processed for user `a5c37b79bf3d40f4a6182cc296585c18`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:28.312080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fcb326a9-fb74-4701-9350-2754e780f5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:28.377517 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:28.463625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 871/1729] 10.222.0.40 () {58 vars in 1057 bytes} [Thu Jun 16 17:34:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:28.467782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc93b75-0406-44dc-a1e1-1e9f01cb9370 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:28.468136 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc93b75-0406-44dc-a1e1-1e9f01cb9370 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:28.468423 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc93b75-0406-44dc-a1e1-1e9f01cb9370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:28.530168 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-8dc93b75-0406-44dc-a1e1-1e9f01cb9370 None None] MFA Rules not processed for user `3583c5993d4c4b3fab50eed455995fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:28.535199 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-8dc93b75-0406-44dc-a1e1-1e9f01cb9370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:28.567141 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 859/1730] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:28.577073 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb326a9-fb74-4701-9350-2754e780f5bf None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:28.739303 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 872/1731] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:28.759907 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da5cb7-1028-4411-a6cb-b9dd4ba3a3a7 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:28.760220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da5cb7-1028-4411-a6cb-b9dd4ba3a3a7 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:28.760510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da5cb7-1028-4411-a6cb-b9dd4ba3a3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:28.796550 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-d1da5cb7-1028-4411-a6cb-b9dd4ba3a3a7 None None] MFA Rules not processed for user `209ada6ba12b4bfda10045b4ee9301f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:28.798007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-d1da5cb7-1028-4411-a6cb-b9dd4ba3a3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:28.812889 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb326a9-fb74-4701-9350-2754e780f5bf None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:28.835799 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9b53ac-81ae-4453-a93e-2800410b950e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:28.839071 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b53ac-81ae-4453-a93e-2800410b950e admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:28.839370 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b53ac-81ae-4453-a93e-2800410b950e admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:28.839658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b53ac-81ae-4453-a93e-2800410b950e admin admin] PATH_INFO: `/v3/projects/168ca52bf9f9465789996df73d0e4633/users/7dabf237bd004fc08c7edc31643562d4/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:28.883113 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9b53ac-81ae-4453-a93e-2800410b950e admin admin] RBAC: Authorizing `identity:create_grant(project_id=168ca52bf9f9465789996df73d0e4633, user_id=7dabf237bd004fc08c7edc31643562d4, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:28.887549 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9b53ac-81ae-4453-a93e-2800410b950e admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:28.928455 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 860/1732] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:28 2022] PUT /identity/v3/projects/168ca52bf9f9465789996df73d0e4633/users/7dabf237bd004fc08c7edc31643562d4/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:28.939245 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9b53ac-81ae-4453-a93e-2800410b950e admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:28.975701 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 873/1733] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:28.979846 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da5cb7-1028-4411-a6cb-b9dd4ba3a3a7 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:29.002054 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da5cb7-1028-4411-a6cb-b9dd4ba3a3a7 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:29.108495 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:29.149644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:29.153056 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9b53ac-81ae-4453-a93e-2800410b950e admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:29.175242 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60012c95-04a0-4f66-86cb-3dfdf75c1d97 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75f0586a8c1941448d750918183f6dc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:29.175932 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60012c95-04a0-4f66-86cb-3dfdf75c1d97 service placement] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:29.176282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60012c95-04a0-4f66-86cb-3dfdf75c1d97 service placement] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:29.176493 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60012c95-04a0-4f66-86cb-3dfdf75c1d97 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:29.185163 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-60012c95-04a0-4f66-86cb-3dfdf75c1d97 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:29.186056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05733594-fb1d-403c-b271-71e77e22fe78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:29.186774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05733594-fb1d-403c-b271-71e77e22fe78 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:29.187068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05733594-fb1d-403c-b271-71e77e22fe78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:29.187358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05733594-fb1d-403c-b271-71e77e22fe78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:29.188150 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05733594-fb1d-403c-b271-71e77e22fe78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:29.190175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05733594-fb1d-403c-b271-71e77e22fe78 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:29.220362 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60012c95-04a0-4f66-86cb-3dfdf75c1d97 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:29.222586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60012c95-04a0-4f66-86cb-3dfdf75c1d97 service placement] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:29.238076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 874/1734] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:28 2022] POST /identity/v3/projects => generated 401 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:29.239474 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05733594-fb1d-403c-b271-71e77e22fe78 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:29.243230 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-05733594-fb1d-403c-b271-71e77e22fe78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:29.259901 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 861/1735] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:29.261525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994272c-207a-4187-b585-78c23d3a3d17 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:29.261824 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994272c-207a-4187-b585-78c23d3a3d17 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:29.262110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994272c-207a-4187-b585-78c23d3a3d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:29.294327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-f994272c-207a-4187-b585-78c23d3a3d17 None None] MFA Rules not processed for user `7dabf237bd004fc08c7edc31643562d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:29.295568 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f994272c-207a-4187-b585-78c23d3a3d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:29.473508 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05733594-fb1d-403c-b271-71e77e22fe78 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:29.479822 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645876fa-fa33-491f-bf15-976535ca092b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75f0586a8c1941448d750918183f6dc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:29.481110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645876fa-fa33-491f-bf15-976535ca092b service placement] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:29.481447 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645876fa-fa33-491f-bf15-976535ca092b service placement] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:29.481782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645876fa-fa33-491f-bf15-976535ca092b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:29.500388 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645876fa-fa33-491f-bf15-976535ca092b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:29.502047 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645876fa-fa33-491f-bf15-976535ca092b service placement] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:29.525629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 862/1736] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:29.527083 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f994272c-207a-4187-b585-78c23d3a3d17 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:29.527228 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 875/1737] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:29.528213 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645876fa-fa33-491f-bf15-976535ca092b service placement] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:29.532583 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f994272c-207a-4187-b585-78c23d3a3d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:29.720098 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f994272c-207a-4187-b585-78c23d3a3d17 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:29.737889 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d13a0-213b-46bf-99ac-8ca135e1621a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:29.740461 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d13a0-213b-46bf-99ac-8ca135e1621a service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:29.742267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d13a0-213b-46bf-99ac-8ca135e1621a service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:29.743771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d13a0-213b-46bf-99ac-8ca135e1621a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:29.773719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645876fa-fa33-491f-bf15-976535ca092b service placement] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:29.789545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc958c-2332-4f74-9006-e9841ce437a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:29.790562 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc958c-2332-4f74-9006-e9841ce437a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:29.791871 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc958c-2332-4f74-9006-e9841ce437a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:29.793705 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc958c-2332-4f74-9006-e9841ce437a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:29.796271 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc958c-2332-4f74-9006-e9841ce437a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:29.798528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc958c-2332-4f74-9006-e9841ce437a2 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:29.849389 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 876/1738] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:29 2022] POST /identity/v3/users => generated 307 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:29.850468 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc958c-2332-4f74-9006-e9841ce437a2 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:29.958131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d13a0-213b-46bf-99ac-8ca135e1621a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:29.963427 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d13a0-213b-46bf-99ac-8ca135e1621a service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:29.969567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc958c-2332-4f74-9006-e9841ce437a2 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:29.986171 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2cc6f-164e-4279-8f62-49ad41fb87b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:29.986866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2cc6f-164e-4279-8f62-49ad41fb87b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:29.987237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2cc6f-164e-4279-8f62-49ad41fb87b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:29.987528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2cc6f-164e-4279-8f62-49ad41fb87b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:29.988265 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2cc6f-164e-4279-8f62-49ad41fb87b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:29.990222 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2cc6f-164e-4279-8f62-49ad41fb87b9 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:29.993013 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 863/1739] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:29.994109 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d13a0-213b-46bf-99ac-8ca135e1621a service nova] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:30.037249 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d13a0-213b-46bf-99ac-8ca135e1621a service nova] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:30.051800 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161874b1-19a8-4472-b219-c47cfdde4784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:30.056364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161874b1-19a8-4472-b219-c47cfdde4784 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:30.057102 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161874b1-19a8-4472-b219-c47cfdde4784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:30.057623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161874b1-19a8-4472-b219-c47cfdde4784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:30.058892 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161874b1-19a8-4472-b219-c47cfdde4784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:30.062856 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161874b1-19a8-4472-b219-c47cfdde4784 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:30.073055 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:30.082159 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 877/1740] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:29 2022] POST /identity/v3/projects => generated 379 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:30.084342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e69898-2021-4dc3-a092-61884500d154 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:30.084342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e69898-2021-4dc3-a092-61884500d154 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:30.084342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e69898-2021-4dc3-a092-61884500d154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:30.101530 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 864/1741] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:30.102637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161874b1-19a8-4472-b219-c47cfdde4784 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:30.156235 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161874b1-19a8-4472-b219-c47cfdde4784 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:30.192719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-b4e69898-2021-4dc3-a092-61884500d154 None None] MFA Rules not processed for user `3faab41439184fecbb3e1021eb81d555`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:30.197672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b4e69898-2021-4dc3-a092-61884500d154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:30.275370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:30.459776 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4a23b-5cdf-4fc5-b2ed-c4e609f25474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:30.460481 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4a23b-5cdf-4fc5-b2ed-c4e609f25474 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:30.460767 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4a23b-5cdf-4fc5-b2ed-c4e609f25474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:30.461503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4a23b-5cdf-4fc5-b2ed-c4e609f25474 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:30.464967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4a23b-5cdf-4fc5-b2ed-c4e609f25474 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:30.492695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4a23b-5cdf-4fc5-b2ed-c4e609f25474 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:30.550094 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 878/1742] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:30.566863 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e69898-2021-4dc3-a092-61884500d154 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:30.610990 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e69898-2021-4dc3-a092-61884500d154 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:30.649204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678095d-af04-4c8a-bbd3-4a7f23357347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:30.650148 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678095d-af04-4c8a-bbd3-4a7f23357347 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:30.650440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678095d-af04-4c8a-bbd3-4a7f23357347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:30.654980 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678095d-af04-4c8a-bbd3-4a7f23357347 admin admin] PATH_INFO: `/v3/projects/07ed5478be314c2591e400490e44a1ae/users/cc2ce5dde2da4bdda8a71445a8706459/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:30.663717 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 865/1743] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:30 2022] POST /identity/v3/users => generated 296 bytes in 561 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:30.665313 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4a23b-5cdf-4fc5-b2ed-c4e609f25474 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:30.678863 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-dac4a23b-5cdf-4fc5-b2ed-c4e609f25474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:30.726213 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678095d-af04-4c8a-bbd3-4a7f23357347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07ed5478be314c2591e400490e44a1ae, user_id=cc2ce5dde2da4bdda8a71445a8706459, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:30.742196 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678095d-af04-4c8a-bbd3-4a7f23357347 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:30.799370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 879/1744] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:30 2022] PUT /identity/v3/projects/07ed5478be314c2591e400490e44a1ae/users/cc2ce5dde2da4bdda8a71445a8706459/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:30.800932 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678095d-af04-4c8a-bbd3-4a7f23357347 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:31.046765 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4a23b-5cdf-4fc5-b2ed-c4e609f25474 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:31.088686 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678095d-af04-4c8a-bbd3-4a7f23357347 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:31.101190 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412962b-f638-4e69-b792-8f835bf3f21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:31.101916 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412962b-f638-4e69-b792-8f835bf3f21b admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:31.102223 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412962b-f638-4e69-b792-8f835bf3f21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:31.102515 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412962b-f638-4e69-b792-8f835bf3f21b admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:31.107238 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c412962b-f638-4e69-b792-8f835bf3f21b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:31.119529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c412962b-f638-4e69-b792-8f835bf3f21b admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:31.155980 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 880/1745] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:31.158031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412962b-f638-4e69-b792-8f835bf3f21b admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:31.221022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412962b-f638-4e69-b792-8f835bf3f21b admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:31.276472 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578bbd7e-2895-4e73-9085-3f29e9d4ce5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:31.277180 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bbd7e-2895-4e73-9085-3f29e9d4ce5c admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:31.277486 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bbd7e-2895-4e73-9085-3f29e9d4ce5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:31.277775 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bbd7e-2895-4e73-9085-3f29e9d4ce5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:31.278648 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578bbd7e-2895-4e73-9085-3f29e9d4ce5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:31.280443 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578bbd7e-2895-4e73-9085-3f29e9d4ce5c admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:31.308909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 881/1746] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:31.310482 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578bbd7e-2895-4e73-9085-3f29e9d4ce5c admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:31.335278 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578bbd7e-2895-4e73-9085-3f29e9d4ce5c admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:31.336973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56889d-86b0-4d74-ae09-f0cc6b3b5d74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:31.338344 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56889d-86b0-4d74-ae09-f0cc6b3b5d74 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:31.338600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56889d-86b0-4d74-ae09-f0cc6b3b5d74 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:31.339119 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56889d-86b0-4d74-ae09-f0cc6b3b5d74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:31.353059 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f3c04-3e92-48cb-9a7f-63c066af17ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:31.354284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f3c04-3e92-48cb-9a7f-63c066af17ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:31.354713 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f3c04-3e92-48cb-9a7f-63c066af17ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:31.355126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f3c04-3e92-48cb-9a7f-63c066af17ef admin admin] PATH_INFO: `/v3/projects/6cf48f50b3c14824a0c6381a72982b73/users/4c93f55f458a4861b3e6018bb2bd2d89/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:31.378984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:31.388579 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f3c04-3e92-48cb-9a7f-63c066af17ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cf48f50b3c14824a0c6381a72982b73, user_id=4c93f55f458a4861b3e6018bb2bd2d89, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:31.391552 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f3c04-3e92-48cb-9a7f-63c066af17ef admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:31.418720 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 882/1747] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:31 2022] PUT /identity/v3/projects/6cf48f50b3c14824a0c6381a72982b73/users/4c93f55f458a4861b3e6018bb2bd2d89/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:31.419944 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f3c04-3e92-48cb-9a7f-63c066af17ef admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:31.447560 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56889d-86b0-4d74-ae09-f0cc6b3b5d74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:31.449312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56889d-86b0-4d74-ae09-f0cc6b3b5d74 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:31.478739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 866/1748] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 814 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:31.480191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56889d-86b0-4d74-ae09-f0cc6b3b5d74 service nova] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:31.515991 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56889d-86b0-4d74-ae09-f0cc6b3b5d74 service nova] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:31.601521 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:31.619804 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f3c04-3e92-48cb-9a7f-63c066af17ef admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:31.633876 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929911f2-ebd1-46ab-84d0-c450d2fbcc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:31.635288 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929911f2-ebd1-46ab-84d0-c450d2fbcc84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:31.636141 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929911f2-ebd1-46ab-84d0-c450d2fbcc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:31.637209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929911f2-ebd1-46ab-84d0-c450d2fbcc84 admin admin] PATH_INFO: `/v3/projects/07ed5478be314c2591e400490e44a1ae/users/cc2ce5dde2da4bdda8a71445a8706459/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:31.641973 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929911f2-ebd1-46ab-84d0-c450d2fbcc84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07ed5478be314c2591e400490e44a1ae, user_id=cc2ce5dde2da4bdda8a71445a8706459, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:31.652379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929911f2-ebd1-46ab-84d0-c450d2fbcc84 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:31.695232 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f9c55-6fda-466a-a22e-54be55695179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:31.695981 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f9c55-6fda-466a-a22e-54be55695179 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:31.698318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f9c55-6fda-466a-a22e-54be55695179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:31.698621 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f9c55-6fda-466a-a22e-54be55695179 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:31.699410 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f9c55-6fda-466a-a22e-54be55695179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:31.703923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f9c55-6fda-466a-a22e-54be55695179 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:31.707498 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 883/1749] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:31 2022] PUT /identity/v3/projects/07ed5478be314c2591e400490e44a1ae/users/cc2ce5dde2da4bdda8a71445a8706459/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:31.728486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 867/1750] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:31.729532 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ac249-b61f-4ba2-8e4a-6916f9f2d9c6 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:31.729650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f9c55-6fda-466a-a22e-54be55695179 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:31.730003 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ac249-b61f-4ba2-8e4a-6916f9f2d9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:31.730956 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ac249-b61f-4ba2-8e4a-6916f9f2d9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:31.749818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f9c55-6fda-466a-a22e-54be55695179 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:31.901841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762848d-fb44-45ef-9d6a-14329f7c4192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:31.902785 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762848d-fb44-45ef-9d6a-14329f7c4192 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:31.903077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762848d-fb44-45ef-9d6a-14329f7c4192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:31.903363 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762848d-fb44-45ef-9d6a-14329f7c4192 admin admin] PATH_INFO: `/v3/projects/6cf48f50b3c14824a0c6381a72982b73/users/4c93f55f458a4861b3e6018bb2bd2d89/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:31.910079 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-4c3ac249-b61f-4ba2-8e4a-6916f9f2d9c6 None None] MFA Rules not processed for user `cc2ce5dde2da4bdda8a71445a8706459`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:31.919350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762848d-fb44-45ef-9d6a-14329f7c4192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cf48f50b3c14824a0c6381a72982b73, user_id=4c93f55f458a4861b3e6018bb2bd2d89, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:31.919954 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4c3ac249-b61f-4ba2-8e4a-6916f9f2d9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:31.937048 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762848d-fb44-45ef-9d6a-14329f7c4192 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:31.994161 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 868/1751] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:31 2022] PUT /identity/v3/projects/6cf48f50b3c14824a0c6381a72982b73/users/4c93f55f458a4861b3e6018bb2bd2d89/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:32.008110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d710080-1b94-46d8-9c01-8afe1293a66a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:32.008411 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d710080-1b94-46d8-9c01-8afe1293a66a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:32.008700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d710080-1b94-46d8-9c01-8afe1293a66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:32.120619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-3d710080-1b94-46d8-9c01-8afe1293a66a None None] MFA Rules not processed for user `4c93f55f458a4861b3e6018bb2bd2d89`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:32.121896 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3d710080-1b94-46d8-9c01-8afe1293a66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:32.223868 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 884/1752] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:31 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 501 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:32.294834 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ac249-b61f-4ba2-8e4a-6916f9f2d9c6 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:32.296876 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4c3ac249-b61f-4ba2-8e4a-6916f9f2d9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:32.307843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:32.424263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 869/1753] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:32.521806 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ac249-b61f-4ba2-8e4a-6916f9f2d9c6 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:32.604279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d710080-1b94-46d8-9c01-8afe1293a66a None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:32.607150 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3d710080-1b94-46d8-9c01-8afe1293a66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:32.631083 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea120a3-3220-4f65-885b-0f2e8f2d17e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:32.631909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea120a3-3220-4f65-885b-0f2e8f2d17e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:32.632013 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea120a3-3220-4f65-885b-0f2e8f2d17e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:32.632256 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea120a3-3220-4f65-885b-0f2e8f2d17e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:32.710850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:32.771023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea120a3-3220-4f65-885b-0f2e8f2d17e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:32.774923 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea120a3-3220-4f65-885b-0f2e8f2d17e9 service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:32.786701 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d710080-1b94-46d8-9c01-8afe1293a66a None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:32.790940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74fe27-c048-41b4-bed7-d290ecd1898b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:32.791786 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74fe27-c048-41b4-bed7-d290ecd1898b service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:32.791894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74fe27-c048-41b4-bed7-d290ecd1898b service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:32.792844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74fe27-c048-41b4-bed7-d290ecd1898b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:32.804106 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 885/1754] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:32 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:32.841525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74fe27-c048-41b4-bed7-d290ecd1898b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:32.843396 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74fe27-c048-41b4-bed7-d290ecd1898b service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:32.878839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 870/1755] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:33.247744 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea120a3-3220-4f65-885b-0f2e8f2d17e9 service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:33.258155 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74fe27-c048-41b4-bed7-d290ecd1898b service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:33.260201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea120a3-3220-4f65-885b-0f2e8f2d17e9 service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:33.274948 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74fe27-c048-41b4-bed7-d290ecd1898b service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:33.287929 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf6517-5e20-402f-b45b-b27af61e5829 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:33.288353 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af514e-b685-4c29-b170-bacc8a089c6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:33.288661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf6517-5e20-402f-b45b-b27af61e5829 service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:33.288955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf6517-5e20-402f-b45b-b27af61e5829 service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:33.289055 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af514e-b685-4c29-b170-bacc8a089c6b service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:33.289273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf6517-5e20-402f-b45b-b27af61e5829 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:33.292904 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af514e-b685-4c29-b170-bacc8a089c6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:33.294984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af514e-b685-4c29-b170-bacc8a089c6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:33.414875 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af514e-b685-4c29-b170-bacc8a089c6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:33.417869 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af514e-b685-4c29-b170-bacc8a089c6b service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:33.470695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 871/1756] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:34:33.541713 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf6517-5e20-402f-b45b-b27af61e5829 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:33.544460 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf6517-5e20-402f-b45b-b27af61e5829 service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:33.578150 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 886/1757] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:33 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:34.045729 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddc8e1-8332-4675-b7d5-832923fb6cc6 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:34.046280 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddc8e1-8332-4675-b7d5-832923fb6cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:34.046280 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddc8e1-8332-4675-b7d5-832923fb6cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:34.118128 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d1ddc8e1-8332-4675-b7d5-832923fb6cc6 None None] MFA Rules not processed for user `cfc2f31f654e486d8505d80aad17d17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:34.120309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d1ddc8e1-8332-4675-b7d5-832923fb6cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:34.408316 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 872/1758] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 17:34:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:34:34.656324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf6517-5e20-402f-b45b-b27af61e5829 service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:34.666128 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-36bf6517-5e20-402f-b45b-b27af61e5829 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:34.948580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf6517-5e20-402f-b45b-b27af61e5829 service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:34.977139 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d01851-da5c-40ef-a41d-734d6488c0e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a37d1201d54afdad9b228ef4f2d787', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:34.978153 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d01851-da5c-40ef-a41d-734d6488c0e9 service swift] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:34.978660 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d01851-da5c-40ef-a41d-734d6488c0e9 service swift] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:34.979152 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d01851-da5c-40ef-a41d-734d6488c0e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:35.006609 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d01851-da5c-40ef-a41d-734d6488c0e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:35.008662 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d01851-da5c-40ef-a41d-734d6488c0e9 service swift] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:35.025806 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 887/1759] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 17:34:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:34:38.033052 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47144f-fc2f-427b-8583-729f739585a8 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:38.033052 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47144f-fc2f-427b-8583-729f739585a8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:38.033052 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47144f-fc2f-427b-8583-729f739585a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:38.172359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-2f47144f-fc2f-427b-8583-729f739585a8 None None] MFA Rules not processed for user `209ada6ba12b4bfda10045b4ee9301f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:38.178711 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2f47144f-fc2f-427b-8583-729f739585a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:38.316533 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:38.631854 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 873/1760] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 601 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:38.994073 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859438c-d90c-409f-979f-90e4cd67269f None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:38.995755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859438c-d90c-409f-979f-90e4cd67269f None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:38.996256 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859438c-d90c-409f-979f-90e4cd67269f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:39.073264 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-b859438c-d90c-409f-979f-90e4cd67269f None None] MFA Rules not processed for user `3faab41439184fecbb3e1021eb81d555`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:39.076111 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b859438c-d90c-409f-979f-90e4cd67269f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:39.180716 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 888/1761] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:40.487582 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d26e8-3fea-4eda-bf26-8fc232615ee7 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:40.488082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d26e8-3fea-4eda-bf26-8fc232615ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:40.488214 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d26e8-3fea-4eda-bf26-8fc232615ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:40.527758 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-410d26e8-3fea-4eda-bf26-8fc232615ee7 None None] MFA Rules not processed for user `a303489493ce4cd2b62e876fbc8a7404`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:40.533552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-410d26e8-3fea-4eda-bf26-8fc232615ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:40.775868 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 874/1762] 10.222.0.47 () {58 vars in 1015 bytes} [Thu Jun 16 17:34:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:40.831341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859438c-d90c-409f-979f-90e4cd67269f None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:40.842463 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d26e8-3fea-4eda-bf26-8fc232615ee7 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:40.873046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859438c-d90c-409f-979f-90e4cd67269f None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:40.891242 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d26e8-3fea-4eda-bf26-8fc232615ee7 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:40.922701 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d9a2e-7acc-491c-be22-cdd5398a4ff3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:40.923720 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d9a2e-7acc-491c-be22-cdd5398a4ff3 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:40.924079 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d9a2e-7acc-491c-be22-cdd5398a4ff3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:40.928088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075c568-c59a-4509-aaa3-c02c40a02e6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:40.928834 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075c568-c59a-4509-aaa3-c02c40a02e6d service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:40.929311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075c568-c59a-4509-aaa3-c02c40a02e6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:40.929683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075c568-c59a-4509-aaa3-c02c40a02e6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:40.933697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d9a2e-7acc-491c-be22-cdd5398a4ff3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:40.994111 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d9a2e-7acc-491c-be22-cdd5398a4ff3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:40.995757 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d9a2e-7acc-491c-be22-cdd5398a4ff3 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:41.010665 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075c568-c59a-4509-aaa3-c02c40a02e6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:41.035236 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075c568-c59a-4509-aaa3-c02c40a02e6d service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:41.089296 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 889/1763] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:41.101965 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d9a2e-7acc-491c-be22-cdd5398a4ff3 service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:41.106824 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-922d9a2e-7acc-491c-be22-cdd5398a4ff3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:41.113553 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 875/1764] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:41.159797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:41.204299 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d9a2e-7acc-491c-be22-cdd5398a4ff3 service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:41.225057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d31dd7c-2c12-4e18-b691-6a379bb0bc4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:41.225770 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31dd7c-2c12-4e18-b691-6a379bb0bc4a service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:41.226066 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31dd7c-2c12-4e18-b691-6a379bb0bc4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:41.226363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31dd7c-2c12-4e18-b691-6a379bb0bc4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:41.318689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d31dd7c-2c12-4e18-b691-6a379bb0bc4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:41.330005 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d31dd7c-2c12-4e18-b691-6a379bb0bc4a service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:41.415344 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 890/1765] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:42.212391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075c568-c59a-4509-aaa3-c02c40a02e6d service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:42.351726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075c568-c59a-4509-aaa3-c02c40a02e6d service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:42.366095 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b7b34-a85d-4d0b-9d1d-35c66f8ca1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:42.368236 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b7b34-a85d-4d0b-9d1d-35c66f8ca1c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:42.369034 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b7b34-a85d-4d0b-9d1d-35c66f8ca1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:42.369748 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b7b34-a85d-4d0b-9d1d-35c66f8ca1c3 admin admin] PATH_INFO: `/v3/users/3583c5993d4c4b3fab50eed455995fc2` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:42.373005 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b7b34-a85d-4d0b-9d1d-35c66f8ca1c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3583c5993d4c4b3fab50eed455995fc2)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:42.376002 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b7b34-a85d-4d0b-9d1d-35c66f8ca1c3 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:42.478795 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a95b7b34-a85d-4d0b-9d1d-35c66f8ca1c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3583c5993d4c4b3fab50eed455995fc2'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:42.544237 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a95b7b34-a85d-4d0b-9d1d-35c66f8ca1c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3583c5993d4c4b3fab50eed455995fc2'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:42.545823 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a95b7b34-a85d-4d0b-9d1d-35c66f8ca1c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3583c5993d4c4b3fab50eed455995fc2'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:42.548845 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:42.559600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a95b7b34-a85d-4d0b-9d1d-35c66f8ca1c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3583c5993d4c4b3fab50eed455995fc2'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:42.577858 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a95b7b34-a85d-4d0b-9d1d-35c66f8ca1c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3583c5993d4c4b3fab50eed455995fc2'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:42.599911 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 876/1766] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:34:42 2022] DELETE /identity/v3/users/3583c5993d4c4b3fab50eed455995fc2 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:42.606772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702e2e5-4dea-4d0b-9863-a9637f0824c4 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:42.607321 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702e2e5-4dea-4d0b-9863-a9637f0824c4 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:42.607532 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702e2e5-4dea-4d0b-9863-a9637f0824c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:42.674666 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-a702e2e5-4dea-4d0b-9863-a9637f0824c4 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:42.679149 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-a702e2e5-4dea-4d0b-9863-a9637f0824c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:42.922512 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 891/1767] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:42.964186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b7b34-a85d-4d0b-9d1d-35c66f8ca1c3 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:42.967626 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a95b7b34-a85d-4d0b-9d1d-35c66f8ca1c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:43.217767 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b7b34-a85d-4d0b-9d1d-35c66f8ca1c3 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:43.248278 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33a2c6-8b26-4a3d-a21f-577aa7794efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:43.251117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33a2c6-8b26-4a3d-a21f-577aa7794efd service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:43.251763 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33a2c6-8b26-4a3d-a21f-577aa7794efd service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:43.255360 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33a2c6-8b26-4a3d-a21f-577aa7794efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:43.314137 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33a2c6-8b26-4a3d-a21f-577aa7794efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:43.315852 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33a2c6-8b26-4a3d-a21f-577aa7794efd service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:43.366726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 877/1768] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:44.056555 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba96ea-8300-4186-ad81-da1fb5b4168e None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:44.057175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba96ea-8300-4186-ad81-da1fb5b4168e None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:44.057175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba96ea-8300-4186-ad81-da1fb5b4168e None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:44.058860 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 892/1769] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 17:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:34:44.073495 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625db24-88fa-4e82-9793-11c04e5651c5 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:44.073847 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625db24-88fa-4e82-9793-11c04e5651c5 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:44.074122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625db24-88fa-4e82-9793-11c04e5651c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:44.168049 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1625db24-88fa-4e82-9793-11c04e5651c5 None None] MFA Rules not processed for user `689a31e26c6b454a91c3fd79678b4d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:44.190278 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1625db24-88fa-4e82-9793-11c04e5651c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:44.530201 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 878/1770] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 17:34:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:45.395651 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba96ea-8300-4186-ad81-da1fb5b4168e None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:45.398320 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-77ba96ea-8300-4186-ad81-da1fb5b4168e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:45.449917 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba96ea-8300-4186-ad81-da1fb5b4168e None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:45.481572 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfcf664-50f3-46bc-b558-a0dea42ad2df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:45.482259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfcf664-50f3-46bc-b558-a0dea42ad2df service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:45.482742 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfcf664-50f3-46bc-b558-a0dea42ad2df service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:45.483311 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfcf664-50f3-46bc-b558-a0dea42ad2df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:45.521567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfcf664-50f3-46bc-b558-a0dea42ad2df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:45.527830 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfcf664-50f3-46bc-b558-a0dea42ad2df service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:45.578667 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 893/1771] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:45.775678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625db24-88fa-4e82-9793-11c04e5651c5 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:45.778908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1625db24-88fa-4e82-9793-11c04e5651c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:45.814373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625db24-88fa-4e82-9793-11c04e5651c5 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:45.846503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdc967-1e45-448e-9e36-41a138fc08f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:45.847713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdc967-1e45-448e-9e36-41a138fc08f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:45.848369 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdc967-1e45-448e-9e36-41a138fc08f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:45.848569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdc967-1e45-448e-9e36-41a138fc08f9 admin admin] PATH_INFO: `/v3/users/7dabf237bd004fc08c7edc31643562d4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:45.852072 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdc967-1e45-448e-9e36-41a138fc08f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dabf237bd004fc08c7edc31643562d4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:45.857665 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdc967-1e45-448e-9e36-41a138fc08f9 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:46.007530 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-68cdc967-1e45-448e-9e36-41a138fc08f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dabf237bd004fc08c7edc31643562d4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:46.050602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-68cdc967-1e45-448e-9e36-41a138fc08f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dabf237bd004fc08c7edc31643562d4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:46.058192 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-68cdc967-1e45-448e-9e36-41a138fc08f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dabf237bd004fc08c7edc31643562d4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:46.079701 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-68cdc967-1e45-448e-9e36-41a138fc08f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dabf237bd004fc08c7edc31643562d4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:46.085024 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:INSERT=1,keystone:DELETE=10 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:46.100177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-68cdc967-1e45-448e-9e36-41a138fc08f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dabf237bd004fc08c7edc31643562d4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:46.112740 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 879/1772] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:34:45 2022] DELETE /identity/v3/users/7dabf237bd004fc08c7edc31643562d4 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:46.118331 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3b6b1-e41f-482d-b8ae-d88ec54259f5 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:46.118896 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3b6b1-e41f-482d-b8ae-d88ec54259f5 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:46.120063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3b6b1-e41f-482d-b8ae-d88ec54259f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:46.159171 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-e8a3b6b1-e41f-482d-b8ae-d88ec54259f5 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:46.161452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-e8a3b6b1-e41f-482d-b8ae-d88ec54259f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:46.338644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 894/1773] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:46.350039 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdc967-1e45-448e-9e36-41a138fc08f9 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:46.351677 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-68cdc967-1e45-448e-9e36-41a138fc08f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:46.565906 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdc967-1e45-448e-9e36-41a138fc08f9 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:46.583589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3e789-8cf1-4c99-966e-5362c290dbd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:46.584470 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3e789-8cf1-4c99-966e-5362c290dbd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:46.584933 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3e789-8cf1-4c99-966e-5362c290dbd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:46.589974 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3e789-8cf1-4c99-966e-5362c290dbd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:46.646242 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3e789-8cf1-4c99-966e-5362c290dbd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:46.647741 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3e789-8cf1-4c99-966e-5362c290dbd5 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:46.707971 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 880/1774] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:48.367114 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ba8c2-61e6-4c89-a9e5-7930ca3544ef None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:48.367982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ba8c2-61e6-4c89-a9e5-7930ca3544ef None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:48.368418 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ba8c2-61e6-4c89-a9e5-7930ca3544ef None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:48.371088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 895/1775] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 17:34:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:34:48.418892 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103c1a0-02dc-42a6-b07b-230190ea77f9 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:48.419724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103c1a0-02dc-42a6-b07b-230190ea77f9 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:48.420618 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103c1a0-02dc-42a6-b07b-230190ea77f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:48.496143 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4103c1a0-02dc-42a6-b07b-230190ea77f9 None None] MFA Rules not processed for user `689a31e26c6b454a91c3fd79678b4d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:48.497822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4103c1a0-02dc-42a6-b07b-230190ea77f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:48.761655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 881/1776] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 17:34:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:49.125179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ba8c2-61e6-4c89-a9e5-7930ca3544ef None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:49.127936 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-f12ba8c2-61e6-4c89-a9e5-7930ca3544ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:49.160751 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ba8c2-61e6-4c89-a9e5-7930ca3544ef None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:49.173633 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb359e-5938-464a-b0c2-440632ed3e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:49.175496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb359e-5938-464a-b0c2-440632ed3e3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:49.175922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb359e-5938-464a-b0c2-440632ed3e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:49.176637 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb359e-5938-464a-b0c2-440632ed3e3f admin admin] PATH_INFO: `/v3/projects/8ae33446bcab410695163c6bb2be7afe` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:49.205141 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb359e-5938-464a-b0c2-440632ed3e3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae33446bcab410695163c6bb2be7afe)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:49.208549 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb359e-5938-464a-b0c2-440632ed3e3f admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:49.364529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c9eb359e-5938-464a-b0c2-440632ed3e3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae33446bcab410695163c6bb2be7afe', 'request_id': 'req-c9eb359e-5938-464a-b0c2-440632ed3e3f'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:49.374401 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c9eb359e-5938-464a-b0c2-440632ed3e3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae33446bcab410695163c6bb2be7afe', 'request_id': 'req-c9eb359e-5938-464a-b0c2-440632ed3e3f'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:49.392957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c9eb359e-5938-464a-b0c2-440632ed3e3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae33446bcab410695163c6bb2be7afe', 'request_id': 'req-c9eb359e-5938-464a-b0c2-440632ed3e3f'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:49.417917 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:49.435846 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c9eb359e-5938-464a-b0c2-440632ed3e3f admin admin] The token cache is being invalidate because project 8ae33446bcab410695163c6bb2be7afe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:34:49.436447 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c9eb359e-5938-464a-b0c2-440632ed3e3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:49.475894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c9eb359e-5938-464a-b0c2-440632ed3e3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:49.479682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 896/1777] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:34:49 2022] DELETE /identity/v3/projects/8ae33446bcab410695163c6bb2be7afe => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:49.483703 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4103c1a0-02dc-42a6-b07b-230190ea77f9 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:49.523633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4103c1a0-02dc-42a6-b07b-230190ea77f9 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:49.551754 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb359e-5938-464a-b0c2-440632ed3e3f admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:49.553445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-c9eb359e-5938-464a-b0c2-440632ed3e3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:49.698610 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde8c58-529d-4fac-a040-4f29cfdf8a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:49.699948 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde8c58-529d-4fac-a040-4f29cfdf8a10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:49.700419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde8c58-529d-4fac-a040-4f29cfdf8a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:49.700895 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde8c58-529d-4fac-a040-4f29cfdf8a10 admin admin] PATH_INFO: `/v3/users/cc2ce5dde2da4bdda8a71445a8706459` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:49.702768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde8c58-529d-4fac-a040-4f29cfdf8a10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc2ce5dde2da4bdda8a71445a8706459)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:49.705935 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde8c58-529d-4fac-a040-4f29cfdf8a10 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:49.750269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:INSERT=1,keystone:DELETE=6,keystone:SELECT=93 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:49.794749 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2cde8c58-529d-4fac-a040-4f29cfdf8a10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc2ce5dde2da4bdda8a71445a8706459'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:49.835125 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2cde8c58-529d-4fac-a040-4f29cfdf8a10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc2ce5dde2da4bdda8a71445a8706459'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:49.837757 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2cde8c58-529d-4fac-a040-4f29cfdf8a10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc2ce5dde2da4bdda8a71445a8706459'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:49.874587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb359e-5938-464a-b0c2-440632ed3e3f admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:49.876382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2cde8c58-529d-4fac-a040-4f29cfdf8a10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc2ce5dde2da4bdda8a71445a8706459'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:49.887447 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613efcc6-f107-4337-a586-ccd5877c7a9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:49.888748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613efcc6-f107-4337-a586-ccd5877c7a9f service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:49.889348 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613efcc6-f107-4337-a586-ccd5877c7a9f service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:49.889866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613efcc6-f107-4337-a586-ccd5877c7a9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:49.892898 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-613efcc6-f107-4337-a586-ccd5877c7a9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:49.901199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2cde8c58-529d-4fac-a040-4f29cfdf8a10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc2ce5dde2da4bdda8a71445a8706459'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:49.921139 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613efcc6-f107-4337-a586-ccd5877c7a9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:49.923483 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 882/1778] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:34:49 2022] DELETE /identity/v3/users/cc2ce5dde2da4bdda8a71445a8706459 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:49.925874 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613efcc6-f107-4337-a586-ccd5877c7a9f service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:49.966978 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 897/1779] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:50.118269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde8c58-529d-4fac-a040-4f29cfdf8a10 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:50.120515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2cde8c58-529d-4fac-a040-4f29cfdf8a10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:50.318436 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde8c58-529d-4fac-a040-4f29cfdf8a10 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:50.333882 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c96eb2-3ba2-4f24-85a2-fa68ba1593fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:50.334986 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c96eb2-3ba2-4f24-85a2-fa68ba1593fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:50.335428 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c96eb2-3ba2-4f24-85a2-fa68ba1593fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:50.335812 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c96eb2-3ba2-4f24-85a2-fa68ba1593fd admin admin] PATH_INFO: `/v3/projects/168ca52bf9f9465789996df73d0e4633` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:50.339558 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c96eb2-3ba2-4f24-85a2-fa68ba1593fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=168ca52bf9f9465789996df73d0e4633)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:50.341675 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c96eb2-3ba2-4f24-85a2-fa68ba1593fd admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:50.434166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-54c96eb2-3ba2-4f24-85a2-fa68ba1593fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '168ca52bf9f9465789996df73d0e4633', 'request_id': 'req-54c96eb2-3ba2-4f24-85a2-fa68ba1593fd'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:50.449313 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-54c96eb2-3ba2-4f24-85a2-fa68ba1593fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '168ca52bf9f9465789996df73d0e4633', 'request_id': 'req-54c96eb2-3ba2-4f24-85a2-fa68ba1593fd'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:50.460253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-54c96eb2-3ba2-4f24-85a2-fa68ba1593fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '168ca52bf9f9465789996df73d0e4633', 'request_id': 'req-54c96eb2-3ba2-4f24-85a2-fa68ba1593fd'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:50.500824 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-54c96eb2-3ba2-4f24-85a2-fa68ba1593fd admin admin] The token cache is being invalidate because project 168ca52bf9f9465789996df73d0e4633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:34:50.501905 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-54c96eb2-3ba2-4f24-85a2-fa68ba1593fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:50.503743 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-54c96eb2-3ba2-4f24-85a2-fa68ba1593fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:50.513750 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 883/1780] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:34:50 2022] DELETE /identity/v3/projects/168ca52bf9f9465789996df73d0e4633 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:50.513982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613efcc6-f107-4337-a586-ccd5877c7a9f service nova] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:50.540528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613efcc6-f107-4337-a586-ccd5877c7a9f service nova] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:50.750100 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c7f1e9-96c6-4423-931d-604be81e6b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:50.750997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7f1e9-96c6-4423-931d-604be81e6b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:50.751302 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7f1e9-96c6-4423-931d-604be81e6b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:50.751655 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7f1e9-96c6-4423-931d-604be81e6b08 admin admin] PATH_INFO: `/v3/users/4c93f55f458a4861b3e6018bb2bd2d89` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:50.756017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c7f1e9-96c6-4423-931d-604be81e6b08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c93f55f458a4861b3e6018bb2bd2d89)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:50.760166 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c7f1e9-96c6-4423-931d-604be81e6b08 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:50.879938 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-62c7f1e9-96c6-4423-931d-604be81e6b08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c93f55f458a4861b3e6018bb2bd2d89'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:50.926846 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-62c7f1e9-96c6-4423-931d-604be81e6b08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c93f55f458a4861b3e6018bb2bd2d89'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:50.926846 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-62c7f1e9-96c6-4423-931d-604be81e6b08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c93f55f458a4861b3e6018bb2bd2d89'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:50.950826 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-62c7f1e9-96c6-4423-931d-604be81e6b08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c93f55f458a4861b3e6018bb2bd2d89'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:50.982745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-62c7f1e9-96c6-4423-931d-604be81e6b08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c93f55f458a4861b3e6018bb2bd2d89'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:51.004004 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 898/1781] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:34:50 2022] DELETE /identity/v3/users/4c93f55f458a4861b3e6018bb2bd2d89 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:34:51.222951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c96eb2-3ba2-4f24-85a2-fa68ba1593fd admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:51.224458 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-54c96eb2-3ba2-4f24-85a2-fa68ba1593fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:51.270271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c96eb2-3ba2-4f24-85a2-fa68ba1593fd admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:51.364136 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:51.389549 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c7f1e9-96c6-4423-931d-604be81e6b08 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:51.391522 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-62c7f1e9-96c6-4423-931d-604be81e6b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:51.403102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:51.417976 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ceac44-5a27-4c36-bb72-a133d3257388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:51.418984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ceac44-5a27-4c36-bb72-a133d3257388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:51.419761 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ceac44-5a27-4c36-bb72-a133d3257388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:51.420417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ceac44-5a27-4c36-bb72-a133d3257388 admin admin] PATH_INFO: `/v3/projects/07ed5478be314c2591e400490e44a1ae` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:51.428106 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ceac44-5a27-4c36-bb72-a133d3257388 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07ed5478be314c2591e400490e44a1ae)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:51.430163 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ceac44-5a27-4c36-bb72-a133d3257388 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:51.553218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c3ceac44-5a27-4c36-bb72-a133d3257388 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07ed5478be314c2591e400490e44a1ae', 'request_id': 'req-c3ceac44-5a27-4c36-bb72-a133d3257388'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:51.587592 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c3ceac44-5a27-4c36-bb72-a133d3257388 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07ed5478be314c2591e400490e44a1ae', 'request_id': 'req-c3ceac44-5a27-4c36-bb72-a133d3257388'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:51.598990 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c3ceac44-5a27-4c36-bb72-a133d3257388 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07ed5478be314c2591e400490e44a1ae', 'request_id': 'req-c3ceac44-5a27-4c36-bb72-a133d3257388'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:51.606073 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c7f1e9-96c6-4423-931d-604be81e6b08 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:51.638605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c3ceac44-5a27-4c36-bb72-a133d3257388 admin admin] The token cache is being invalidate because project 07ed5478be314c2591e400490e44a1ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:34:51.641343 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c3ceac44-5a27-4c36-bb72-a133d3257388 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:51.642280 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c3ceac44-5a27-4c36-bb72-a133d3257388 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:51.649473 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 884/1782] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:34:51 2022] DELETE /identity/v3/projects/07ed5478be314c2591e400490e44a1ae => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:51.657792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9853d03-b765-46c0-8cd1-4c56a4b9d6a8 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:51.658202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9853d03-b765-46c0-8cd1-4c56a4b9d6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:51.659191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9853d03-b765-46c0-8cd1-4c56a4b9d6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:51.697032 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-f9853d03-b765-46c0-8cd1-4c56a4b9d6a8 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:51.698418 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f9853d03-b765-46c0-8cd1-4c56a4b9d6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:51.803748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965f6b3-4ef5-4930-b562-c760aa87a2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:51.805084 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965f6b3-4ef5-4930-b562-c760aa87a2de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:51.805558 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965f6b3-4ef5-4930-b562-c760aa87a2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:51.805993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965f6b3-4ef5-4930-b562-c760aa87a2de admin admin] PATH_INFO: `/v3/users/c3864bcfbb384b8c8dbdf8bb63b8074a` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:51.808180 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965f6b3-4ef5-4930-b562-c760aa87a2de admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3864bcfbb384b8c8dbdf8bb63b8074a)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:51.810513 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 885/1783] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:51.812601 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965f6b3-4ef5-4930-b562-c760aa87a2de admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:51.818666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647849f-d5c7-4327-b34a-c3d71c6858ee None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:51.818969 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647849f-d5c7-4327-b34a-c3d71c6858ee None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:51.819253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647849f-d5c7-4327-b34a-c3d71c6858ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:51.863406 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4647849f-d5c7-4327-b34a-c3d71c6858ee None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:51.864806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4647849f-d5c7-4327-b34a-c3d71c6858ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:51.886102 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 886/1784] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:51.890332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647849f-d5c7-4327-b34a-c3d71c6858ee None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:51.904061 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-6965f6b3-4ef5-4930-b562-c760aa87a2de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3864bcfbb384b8c8dbdf8bb63b8074a'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:51.938765 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-6965f6b3-4ef5-4930-b562-c760aa87a2de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3864bcfbb384b8c8dbdf8bb63b8074a'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:51.939948 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-6965f6b3-4ef5-4930-b562-c760aa87a2de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3864bcfbb384b8c8dbdf8bb63b8074a'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:51.945899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647849f-d5c7-4327-b34a-c3d71c6858ee None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:51.962702 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-6965f6b3-4ef5-4930-b562-c760aa87a2de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3864bcfbb384b8c8dbdf8bb63b8074a'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:51.966456 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571a77c-b622-4a86-8fbc-4dcdc39bbd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:51.967059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571a77c-b622-4a86-8fbc-4dcdc39bbd70 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:51.967354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571a77c-b622-4a86-8fbc-4dcdc39bbd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:51.967636 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571a77c-b622-4a86-8fbc-4dcdc39bbd70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:51.968394 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571a77c-b622-4a86-8fbc-4dcdc39bbd70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:51.968708 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571a77c-b622-4a86-8fbc-4dcdc39bbd70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:51.971549 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571a77c-b622-4a86-8fbc-4dcdc39bbd70 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:51.994521 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-6965f6b3-4ef5-4930-b562-c760aa87a2de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3864bcfbb384b8c8dbdf8bb63b8074a'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:52.004752 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 887/1785] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:34:52.005931 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571a77c-b622-4a86-8fbc-4dcdc39bbd70 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:52.007703 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8571a77c-b622-4a86-8fbc-4dcdc39bbd70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:52.012529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 899/1786] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:34:51 2022] DELETE /identity/v3/users/c3864bcfbb384b8c8dbdf8bb63b8074a => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:52.017271 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468cd55-8c23-4e80-9d2c-ee3094079404 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:52.018223 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468cd55-8c23-4e80-9d2c-ee3094079404 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:52.018783 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468cd55-8c23-4e80-9d2c-ee3094079404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:52.023464 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571a77c-b622-4a86-8fbc-4dcdc39bbd70 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:52.046514 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-7468cd55-8c23-4e80-9d2c-ee3094079404 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:52.047980 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-7468cd55-8c23-4e80-9d2c-ee3094079404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:52.102930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:52.119095 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:52.143583 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470326f-c376-4e64-9778-d42b3affe647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:52.144554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470326f-c376-4e64-9778-d42b3affe647 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:52.145004 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470326f-c376-4e64-9778-d42b3affe647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:52.145453 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470326f-c376-4e64-9778-d42b3affe647 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:52.146310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f470326f-c376-4e64-9778-d42b3affe647 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:52.149003 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f470326f-c376-4e64-9778-d42b3affe647 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:52.188217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 888/1787] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:52 2022] POST /identity/v3/projects => generated 409 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:52.189021 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470326f-c376-4e64-9778-d42b3affe647 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:52.193419 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 900/1788] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:34:52.196890 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f470326f-c376-4e64-9778-d42b3affe647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:52.196989 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468cd55-8c23-4e80-9d2c-ee3094079404 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:52.208011 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468cd55-8c23-4e80-9d2c-ee3094079404 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:52.287413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4fcf9-d663-4130-a01a-56c2a74fe18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:52.288336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4fcf9-d663-4130-a01a-56c2a74fe18e admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:52.289343 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4fcf9-d663-4130-a01a-56c2a74fe18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:52.289735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4fcf9-d663-4130-a01a-56c2a74fe18e admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:52.291041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d4fcf9-d663-4130-a01a-56c2a74fe18e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:52.293547 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d4fcf9-d663-4130-a01a-56c2a74fe18e admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:52.318094 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 901/1789] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:52 2022] POST /identity/v3/users => generated 305 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:52.319575 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4fcf9-d663-4130-a01a-56c2a74fe18e admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:52.322080 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-c0d4fcf9-d663-4130-a01a-56c2a74fe18e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:52.324705 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470326f-c376-4e64-9778-d42b3affe647 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:52.329623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d581ed-be3b-463c-b643-2f826efcf7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:52.331116 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d581ed-be3b-463c-b643-2f826efcf7d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:52.331467 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d581ed-be3b-463c-b643-2f826efcf7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:52.331837 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d581ed-be3b-463c-b643-2f826efcf7d7 admin admin] PATH_INFO: `/v3/projects/6cf48f50b3c14824a0c6381a72982b73` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:52.334190 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d581ed-be3b-463c-b643-2f826efcf7d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cf48f50b3c14824a0c6381a72982b73)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:52.336465 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d581ed-be3b-463c-b643-2f826efcf7d7 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:52.415118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4fcf9-d663-4130-a01a-56c2a74fe18e admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:52.422009 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e1d581ed-be3b-463c-b643-2f826efcf7d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cf48f50b3c14824a0c6381a72982b73', 'request_id': 'req-e1d581ed-be3b-463c-b643-2f826efcf7d7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:52.435836 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e1d581ed-be3b-463c-b643-2f826efcf7d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cf48f50b3c14824a0c6381a72982b73', 'request_id': 'req-e1d581ed-be3b-463c-b643-2f826efcf7d7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:52.445098 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e1d581ed-be3b-463c-b643-2f826efcf7d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cf48f50b3c14824a0c6381a72982b73', 'request_id': 'req-e1d581ed-be3b-463c-b643-2f826efcf7d7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:52.462550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e1d581ed-be3b-463c-b643-2f826efcf7d7 admin admin] The token cache is being invalidate because project 6cf48f50b3c14824a0c6381a72982b73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:34:52.462883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e1d581ed-be3b-463c-b643-2f826efcf7d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:52.463783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e1d581ed-be3b-463c-b643-2f826efcf7d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:52.470167 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 889/1790] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:34:52 2022] DELETE /identity/v3/projects/6cf48f50b3c14824a0c6381a72982b73 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:52.470167 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d581ed-be3b-463c-b643-2f826efcf7d7 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:52.470167 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e1d581ed-be3b-463c-b643-2f826efcf7d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:52.483167 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475979c8-d2e4-43bf-8729-ed9d1c4d5aae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:52.484556 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475979c8-d2e4-43bf-8729-ed9d1c4d5aae service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:52.485048 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475979c8-d2e4-43bf-8729-ed9d1c4d5aae service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:52.485441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475979c8-d2e4-43bf-8729-ed9d1c4d5aae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:52.487808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-475979c8-d2e4-43bf-8729-ed9d1c4d5aae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:52.522966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:52.562847 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475979c8-d2e4-43bf-8729-ed9d1c4d5aae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:52.564453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475979c8-d2e4-43bf-8729-ed9d1c4d5aae service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:52.567610 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d581ed-be3b-463c-b643-2f826efcf7d7 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:52.576284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 902/1791] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:52.577728 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581faa65-791e-4336-b72f-bc362336a2e9 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:52.578218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581faa65-791e-4336-b72f-bc362336a2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:52.578525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ec11f-0801-45bc-8015-6cd653e6d6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:52.578666 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581faa65-791e-4336-b72f-bc362336a2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:52.579329 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ec11f-0801-45bc-8015-6cd653e6d6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:52.579520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ec11f-0801-45bc-8015-6cd653e6d6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:52.580347 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ec11f-0801-45bc-8015-6cd653e6d6d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:52.581353 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ec11f-0801-45bc-8015-6cd653e6d6d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:52.584120 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ec11f-0801-45bc-8015-6cd653e6d6d2 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:52.587774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:52.600416 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-581faa65-791e-4336-b72f-bc362336a2e9 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:52.601882 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-581faa65-791e-4336-b72f-bc362336a2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:52.607492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 890/1792] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:52.611869 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ec11f-0801-45bc-8015-6cd653e6d6d2 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:52.616299 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 903/1793] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:52.633774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2038e-696c-4242-8def-7e20bdf237a7 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:52.633774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2038e-696c-4242-8def-7e20bdf237a7 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:52.633774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2038e-696c-4242-8def-7e20bdf237a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:52.634117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ec11f-0801-45bc-8015-6cd653e6d6d2 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:52.634117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4bf39-1ecf-4faa-88c5-0f6660e8f073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:52.634117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4bf39-1ecf-4faa-88c5-0f6660e8f073 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:52.635179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4bf39-1ecf-4faa-88c5-0f6660e8f073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:52.635562 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4bf39-1ecf-4faa-88c5-0f6660e8f073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:52.637141 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4bf39-1ecf-4faa-88c5-0f6660e8f073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:52.639203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4bf39-1ecf-4faa-88c5-0f6660e8f073 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:52.650060 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-3ec2038e-696c-4242-8def-7e20bdf237a7 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:52.651318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-3ec2038e-696c-4242-8def-7e20bdf237a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:52.665514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 891/1794] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:52.666788 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 904/1795] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:52.670600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4bf39-1ecf-4faa-88c5-0f6660e8f073 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:52.670844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2038e-696c-4242-8def-7e20bdf237a7 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:52.682536 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4bf39-1ecf-4faa-88c5-0f6660e8f073 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:52.686843 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2038e-696c-4242-8def-7e20bdf237a7 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:52.690985 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225232bf-7117-44d0-8621-66fb2aac5ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:52.692155 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225232bf-7117-44d0-8621-66fb2aac5ce1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:52.692444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225232bf-7117-44d0-8621-66fb2aac5ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:52.692723 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225232bf-7117-44d0-8621-66fb2aac5ce1 admin admin] PATH_INFO: `/v3/projects/64a0889577c9418ab0af5a10ccc89e79/users/0c678bcb17de4c65ba373d327edde6bc/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:52.695635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b20dd4-2693-4b80-b43b-a2f32c44c02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:52.696683 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b20dd4-2693-4b80-b43b-a2f32c44c02f admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:52.697081 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b20dd4-2693-4b80-b43b-a2f32c44c02f admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:52.698432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b20dd4-2693-4b80-b43b-a2f32c44c02f admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:52.699568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b20dd4-2693-4b80-b43b-a2f32c44c02f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:52.700099 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b20dd4-2693-4b80-b43b-a2f32c44c02f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:52.702164 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b20dd4-2693-4b80-b43b-a2f32c44c02f admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:52.718598 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225232bf-7117-44d0-8621-66fb2aac5ce1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64a0889577c9418ab0af5a10ccc89e79, user_id=0c678bcb17de4c65ba373d327edde6bc, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:52.729981 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 905/1796] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:34:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:34:52.733531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225232bf-7117-44d0-8621-66fb2aac5ce1 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:52.735912 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b20dd4-2693-4b80-b43b-a2f32c44c02f admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:52.757566 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 892/1797] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:52 2022] PUT /identity/v3/projects/64a0889577c9418ab0af5a10ccc89e79/users/0c678bcb17de4c65ba373d327edde6bc/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:52.762495 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b20dd4-2693-4b80-b43b-a2f32c44c02f admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:52.769503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797b788-c0c4-41a0-b0da-e88a2cb79b0b None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:52.769834 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797b788-c0c4-41a0-b0da-e88a2cb79b0b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:52.770119 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797b788-c0c4-41a0-b0da-e88a2cb79b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:52.808786 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-3797b788-c0c4-41a0-b0da-e88a2cb79b0b None None] MFA Rules not processed for user `0c678bcb17de4c65ba373d327edde6bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:52.811063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3797b788-c0c4-41a0-b0da-e88a2cb79b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:52.868704 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690c1fc-6f19-4d4d-891f-20d2d8183df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:52.868704 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690c1fc-6f19-4d4d-891f-20d2d8183df0 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:52.868704 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690c1fc-6f19-4d4d-891f-20d2d8183df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:52.868704 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690c1fc-6f19-4d4d-891f-20d2d8183df0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:52.869261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690c1fc-6f19-4d4d-891f-20d2d8183df0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:52.875333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690c1fc-6f19-4d4d-891f-20d2d8183df0 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:52.935089 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 906/1798] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:52 2022] POST /identity/v3/projects => generated 387 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:52.941327 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690c1fc-6f19-4d4d-891f-20d2d8183df0 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:52.956532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 893/1799] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:52 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:52.960041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797b788-c0c4-41a0-b0da-e88a2cb79b0b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:52.986238 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797b788-c0c4-41a0-b0da-e88a2cb79b0b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:53.085352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690c1fc-6f19-4d4d-891f-20d2d8183df0 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:53.092157 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db436f1c-5cf8-4fd6-9f33-a0084eca094a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:53.093748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db436f1c-5cf8-4fd6-9f33-a0084eca094a admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:53.094269 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db436f1c-5cf8-4fd6-9f33-a0084eca094a admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:53.094770 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db436f1c-5cf8-4fd6-9f33-a0084eca094a admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:53.096987 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db436f1c-5cf8-4fd6-9f33-a0084eca094a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:53.100228 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db436f1c-5cf8-4fd6-9f33-a0084eca094a admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:53.141845 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a23764-ba5a-4176-9f70-f3d1d4762371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:53.144332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a23764-ba5a-4176-9f70-f3d1d4762371 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:53.144842 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a23764-ba5a-4176-9f70-f3d1d4762371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:53.146081 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 907/1800] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:52 2022] POST /identity/v3/users => generated 294 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:53.146652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a23764-ba5a-4176-9f70-f3d1d4762371 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:53.147559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a23764-ba5a-4176-9f70-f3d1d4762371 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:53.150567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db436f1c-5cf8-4fd6-9f33-a0084eca094a admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:53.154868 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a23764-ba5a-4176-9f70-f3d1d4762371 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:53.169683 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db436f1c-5cf8-4fd6-9f33-a0084eca094a admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:53.178132 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f189e9-ac17-4f93-bfb4-d65a68e8e83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:53.179047 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f189e9-ac17-4f93-bfb4-d65a68e8e83c admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:53.179748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f189e9-ac17-4f93-bfb4-d65a68e8e83c admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:53.179748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f189e9-ac17-4f93-bfb4-d65a68e8e83c admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:53.181183 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f189e9-ac17-4f93-bfb4-d65a68e8e83c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:53.183370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f189e9-ac17-4f93-bfb4-d65a68e8e83c admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:53.204575 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 908/1801] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:53 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:53.208688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f189e9-ac17-4f93-bfb4-d65a68e8e83c admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:53.218128 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:53.224536 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 894/1802] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:52 2022] POST /identity/v3/projects => generated 409 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:34:53.225662 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a23764-ba5a-4176-9f70-f3d1d4762371 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:53.229629 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f189e9-ac17-4f93-bfb4-d65a68e8e83c admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:53.298577 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:53.340020 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a23764-ba5a-4176-9f70-f3d1d4762371 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:53.347319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873676f6-dcb0-47d5-94c5-03e9f5130af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:53.348193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873676f6-dcb0-47d5-94c5-03e9f5130af5 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:53.348533 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873676f6-dcb0-47d5-94c5-03e9f5130af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:53.348823 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873676f6-dcb0-47d5-94c5-03e9f5130af5 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:53.349656 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873676f6-dcb0-47d5-94c5-03e9f5130af5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:53.351678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873676f6-dcb0-47d5-94c5-03e9f5130af5 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:53.369451 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4407f-b7b0-4fce-bf9c-35b82e545a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:53.380071 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4407f-b7b0-4fce-bf9c-35b82e545a0a admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:53.384214 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4407f-b7b0-4fce-bf9c-35b82e545a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:53.384780 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4407f-b7b0-4fce-bf9c-35b82e545a0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:53.385862 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4407f-b7b0-4fce-bf9c-35b82e545a0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:53.388660 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 895/1803] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:53 2022] POST /identity/v3/users => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:53.389238 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873676f6-dcb0-47d5-94c5-03e9f5130af5 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:53.400402 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-873676f6-dcb0-47d5-94c5-03e9f5130af5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:53.402425 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4407f-b7b0-4fce-bf9c-35b82e545a0a admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:53.432610 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 909/1804] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:53 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:34:53.436264 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4407f-b7b0-4fce-bf9c-35b82e545a0a admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:53.462021 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873676f6-dcb0-47d5-94c5-03e9f5130af5 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:53.464503 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4407f-b7b0-4fce-bf9c-35b82e545a0a admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:53.469483 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7602f-4d73-4591-bb32-8b03771df7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:53.471644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7602f-4d73-4591-bb32-8b03771df7a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:53.471944 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7602f-4d73-4591-bb32-8b03771df7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:53.472234 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7602f-4d73-4591-bb32-8b03771df7a6 admin admin] PATH_INFO: `/v3/projects/3055a17e0b6c4d579559f55e949a98a9` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:53.478854 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecacc36d-1a79-4f0a-afbc-de4004b41e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:53.482632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacc36d-1a79-4f0a-afbc-de4004b41e1a admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:53.483297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacc36d-1a79-4f0a-afbc-de4004b41e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:53.483778 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacc36d-1a79-4f0a-afbc-de4004b41e1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:53.484703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecacc36d-1a79-4f0a-afbc-de4004b41e1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:53.488107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7602f-4d73-4591-bb32-8b03771df7a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3055a17e0b6c4d579559f55e949a98a9)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:53.488448 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecacc36d-1a79-4f0a-afbc-de4004b41e1a admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:53.490749 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7602f-4d73-4591-bb32-8b03771df7a6 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:53.513302 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 910/1805] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:53 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:53.513302 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecacc36d-1a79-4f0a-afbc-de4004b41e1a admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:53.525795 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecacc36d-1a79-4f0a-afbc-de4004b41e1a admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:53.531914 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f188515-8005-4714-a44a-5e0557e70865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:53.533940 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f188515-8005-4714-a44a-5e0557e70865 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:53.534289 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f188515-8005-4714-a44a-5e0557e70865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:53.534586 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f188515-8005-4714-a44a-5e0557e70865 admin admin] PATH_INFO: `/v3/projects/5fbd68c034e54a81b0c6dac3aba5ae83/users/d78bc4d2d3f04f1984d485691a384c3b/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:53.555553 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f188515-8005-4714-a44a-5e0557e70865 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fbd68c034e54a81b0c6dac3aba5ae83, user_id=d78bc4d2d3f04f1984d485691a384c3b, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:53.558284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f188515-8005-4714-a44a-5e0557e70865 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:53.560199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-03b7602f-4d73-4591-bb32-8b03771df7a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3055a17e0b6c4d579559f55e949a98a9', 'request_id': 'req-03b7602f-4d73-4591-bb32-8b03771df7a6'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:53.570555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-03b7602f-4d73-4591-bb32-8b03771df7a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3055a17e0b6c4d579559f55e949a98a9', 'request_id': 'req-03b7602f-4d73-4591-bb32-8b03771df7a6'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:53.574522 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 911/1806] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:53 2022] PUT /identity/v3/projects/5fbd68c034e54a81b0c6dac3aba5ae83/users/d78bc4d2d3f04f1984d485691a384c3b/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:53.578570 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f188515-8005-4714-a44a-5e0557e70865 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:53.579014 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-03b7602f-4d73-4591-bb32-8b03771df7a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3055a17e0b6c4d579559f55e949a98a9', 'request_id': 'req-03b7602f-4d73-4591-bb32-8b03771df7a6'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:53.591928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-03b7602f-4d73-4591-bb32-8b03771df7a6 admin admin] The token cache is being invalidate because project 3055a17e0b6c4d579559f55e949a98a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:34:53.592288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-03b7602f-4d73-4591-bb32-8b03771df7a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:53.592894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-03b7602f-4d73-4591-bb32-8b03771df7a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:53.594545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 896/1807] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:34:53 2022] DELETE /identity/v3/projects/3055a17e0b6c4d579559f55e949a98a9 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:53.596360 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d25e09-692d-4f6b-8603-730ed7855821 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:53.596658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d25e09-692d-4f6b-8603-730ed7855821 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:53.596924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d25e09-692d-4f6b-8603-730ed7855821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:53.632589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-82d25e09-692d-4f6b-8603-730ed7855821 None None] MFA Rules not processed for user `d78bc4d2d3f04f1984d485691a384c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:53.633858 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-82d25e09-692d-4f6b-8603-730ed7855821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:53.668766 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f188515-8005-4714-a44a-5e0557e70865 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:53.674930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c3ac6-f259-4672-b38a-cffe04247d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:53.678135 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c3ac6-f259-4672-b38a-cffe04247d13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:53.679091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c3ac6-f259-4672-b38a-cffe04247d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:53.679091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c3ac6-f259-4672-b38a-cffe04247d13 admin admin] PATH_INFO: `/v3/projects/fa2168e20e0d4084b17b331d1e1b019a/users/ce19cc3b78d14dcb8b6cc27a9a1caf3e/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:53.700963 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c3ac6-f259-4672-b38a-cffe04247d13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa2168e20e0d4084b17b331d1e1b019a, user_id=ce19cc3b78d14dcb8b6cc27a9a1caf3e, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:53.703689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c3ac6-f259-4672-b38a-cffe04247d13 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:53.709672 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 897/1808] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:53.714121 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d25e09-692d-4f6b-8603-730ed7855821 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:53.716081 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-82d25e09-692d-4f6b-8603-730ed7855821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:53.723100 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 912/1809] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:53 2022] PUT /identity/v3/projects/fa2168e20e0d4084b17b331d1e1b019a/users/ce19cc3b78d14dcb8b6cc27a9a1caf3e/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:53.723879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c3ac6-f259-4672-b38a-cffe04247d13 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:53.725654 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-797c3ac6-f259-4672-b38a-cffe04247d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:53.734399 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d25e09-692d-4f6b-8603-730ed7855821 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:53.787592 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:53.812899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957fd3c-36d1-4ce9-ab61-b40057e82155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:53.814797 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957fd3c-36d1-4ce9-ab61-b40057e82155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:53.815606 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957fd3c-36d1-4ce9-ab61-b40057e82155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:53.815606 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957fd3c-36d1-4ce9-ab61-b40057e82155 admin admin] PATH_INFO: `/v3/users/a6f128138b9a4a748ba7203581a7e6c1` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:53.817535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957fd3c-36d1-4ce9-ab61-b40057e82155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6f128138b9a4a748ba7203581a7e6c1)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:53.819769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957fd3c-36d1-4ce9-ab61-b40057e82155 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:53.902536 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1957fd3c-36d1-4ce9-ab61-b40057e82155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6f128138b9a4a748ba7203581a7e6c1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:53.918525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1957fd3c-36d1-4ce9-ab61-b40057e82155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6f128138b9a4a748ba7203581a7e6c1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:53.919612 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1957fd3c-36d1-4ce9-ab61-b40057e82155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6f128138b9a4a748ba7203581a7e6c1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:53.928954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1957fd3c-36d1-4ce9-ab61-b40057e82155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6f128138b9a4a748ba7203581a7e6c1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:53.942242 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1957fd3c-36d1-4ce9-ab61-b40057e82155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6f128138b9a4a748ba7203581a7e6c1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:53.962863 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 898/1810] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:34:53 2022] DELETE /identity/v3/users/a6f128138b9a4a748ba7203581a7e6c1 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:53.963707 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957fd3c-36d1-4ce9-ab61-b40057e82155 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:53.968737 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1957fd3c-36d1-4ce9-ab61-b40057e82155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:54.019369 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:54.056368 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c3ac6-f259-4672-b38a-cffe04247d13 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:54.079646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957fd3c-36d1-4ce9-ab61-b40057e82155 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:54.092853 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb5772-f178-487a-ac44-c86f738065eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:54.094048 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb5772-f178-487a-ac44-c86f738065eb admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:54.094343 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb5772-f178-487a-ac44-c86f738065eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:54.094628 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb5772-f178-487a-ac44-c86f738065eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:54.095440 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdb5772-f178-487a-ac44-c86f738065eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:54.097782 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdb5772-f178-487a-ac44-c86f738065eb admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:54.118726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 899/1811] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:53 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:54.124704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb5772-f178-487a-ac44-c86f738065eb admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:54.145607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb5772-f178-487a-ac44-c86f738065eb admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:54.159301 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68671ca-ef58-46d3-b0fb-3cf4a5d33c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:54.159715 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68671ca-ef58-46d3-b0fb-3cf4a5d33c81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:54.160064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68671ca-ef58-46d3-b0fb-3cf4a5d33c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:54.160348 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68671ca-ef58-46d3-b0fb-3cf4a5d33c81 admin admin] PATH_INFO: `/v3/projects/fa2168e20e0d4084b17b331d1e1b019a/users/ce19cc3b78d14dcb8b6cc27a9a1caf3e/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:54.164905 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68671ca-ef58-46d3-b0fb-3cf4a5d33c81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa2168e20e0d4084b17b331d1e1b019a, user_id=ce19cc3b78d14dcb8b6cc27a9a1caf3e, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:54.170931 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68671ca-ef58-46d3-b0fb-3cf4a5d33c81 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:54.200611 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 900/1812] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:54 2022] PUT /identity/v3/projects/fa2168e20e0d4084b17b331d1e1b019a/users/ce19cc3b78d14dcb8b6cc27a9a1caf3e/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:54.213270 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e664806-e60d-45a3-92b9-4b975b7bc957 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:54.213644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e664806-e60d-45a3-92b9-4b975b7bc957 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:54.213848 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e664806-e60d-45a3-92b9-4b975b7bc957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:54.278314 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8def8-6e37-424a-86a6-1f6a2c351a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:54.281036 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8def8-6e37-424a-86a6-1f6a2c351a42 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:54.281185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-8e664806-e60d-45a3-92b9-4b975b7bc957 None None] MFA Rules not processed for user `ce19cc3b78d14dcb8b6cc27a9a1caf3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:54.281826 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8def8-6e37-424a-86a6-1f6a2c351a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:54.284030 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8e664806-e60d-45a3-92b9-4b975b7bc957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:54.285798 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8def8-6e37-424a-86a6-1f6a2c351a42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:54.287661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8def8-6e37-424a-86a6-1f6a2c351a42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:54.291668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8def8-6e37-424a-86a6-1f6a2c351a42 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:54.355217 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 913/1813] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:53 2022] POST /identity/v3/projects => generated 385 bytes in 632 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:54.360512 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8def8-6e37-424a-86a6-1f6a2c351a42 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:54.362801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-9df8def8-6e37-424a-86a6-1f6a2c351a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:54.384549 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:54.463381 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 901/1814] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:54 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:54.480140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e664806-e60d-45a3-92b9-4b975b7bc957 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:54.482181 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8e664806-e60d-45a3-92b9-4b975b7bc957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:54.490269 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8def8-6e37-424a-86a6-1f6a2c351a42 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:54.497102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2f5ba-8c72-43d3-aa8d-c993ed8c1d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:54.498472 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2f5ba-8c72-43d3-aa8d-c993ed8c1d22 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:54.498767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2f5ba-8c72-43d3-aa8d-c993ed8c1d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:54.499052 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2f5ba-8c72-43d3-aa8d-c993ed8c1d22 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:54.499765 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2f5ba-8c72-43d3-aa8d-c993ed8c1d22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:54.503239 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2f5ba-8c72-43d3-aa8d-c993ed8c1d22 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:54.547638 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 914/1815] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:54 2022] POST /identity/v3/users => generated 299 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:54.555501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2f5ba-8c72-43d3-aa8d-c993ed8c1d22 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:54.572630 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2f5ba-8c72-43d3-aa8d-c993ed8c1d22 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:54.581135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e664806-e60d-45a3-92b9-4b975b7bc957 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:54.582931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201c611-f126-4c41-929e-8026c308d7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:54.583649 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201c611-f126-4c41-929e-8026c308d7f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:54.583998 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201c611-f126-4c41-929e-8026c308d7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:54.584294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201c611-f126-4c41-929e-8026c308d7f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:54.585049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201c611-f126-4c41-929e-8026c308d7f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:54.587238 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201c611-f126-4c41-929e-8026c308d7f6 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:54.608543 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 915/1816] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:54.613083 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201c611-f126-4c41-929e-8026c308d7f6 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:54.623767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201c611-f126-4c41-929e-8026c308d7f6 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:54.629557 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957385b-9547-4d62-907f-1fb5f0ce874b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:54.630460 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957385b-9547-4d62-907f-1fb5f0ce874b admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:54.631147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957385b-9547-4d62-907f-1fb5f0ce874b admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:54.631147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957385b-9547-4d62-907f-1fb5f0ce874b admin admin] PATH_INFO: `/v3/projects/c21dab3cf7bf4e03848ba882ca7f53c8/users/0eb0b50334474fae82659549c510c1f5/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:54.652635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957385b-9547-4d62-907f-1fb5f0ce874b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c21dab3cf7bf4e03848ba882ca7f53c8, user_id=0eb0b50334474fae82659549c510c1f5, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:54.655509 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957385b-9547-4d62-907f-1fb5f0ce874b admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:54.669808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd227ac-caa4-47bb-ad48-21cfc2f028d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:54.670875 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd227ac-caa4-47bb-ad48-21cfc2f028d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:54.671167 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd227ac-caa4-47bb-ad48-21cfc2f028d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:54.671484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd227ac-caa4-47bb-ad48-21cfc2f028d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:54.680667 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 916/1817] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:54 2022] PUT /identity/v3/projects/c21dab3cf7bf4e03848ba882ca7f53c8/users/0eb0b50334474fae82659549c510c1f5/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:34:54.681726 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957385b-9547-4d62-907f-1fb5f0ce874b admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:54.683480 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b957385b-9547-4d62-907f-1fb5f0ce874b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:54.730315 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:54.749470 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd227ac-caa4-47bb-ad48-21cfc2f028d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:54.751036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd227ac-caa4-47bb-ad48-21cfc2f028d9 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:54.755949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957385b-9547-4d62-907f-1fb5f0ce874b admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:54.760656 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39195c37-dbbf-4cda-998c-5c7b3407acab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:54.761552 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39195c37-dbbf-4cda-998c-5c7b3407acab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:54.761841 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39195c37-dbbf-4cda-998c-5c7b3407acab admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:54.762119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39195c37-dbbf-4cda-998c-5c7b3407acab admin admin] PATH_INFO: `/v3/projects/0ba32a55e31d4a3b972c75e35236ce99` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:54.762598 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 902/1818] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:54 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:34:54.763497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd227ac-caa4-47bb-ad48-21cfc2f028d9 service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:54.773104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39195c37-dbbf-4cda-998c-5c7b3407acab admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ba32a55e31d4a3b972c75e35236ce99)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:54.775097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39195c37-dbbf-4cda-998c-5c7b3407acab admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:54.775224 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd227ac-caa4-47bb-ad48-21cfc2f028d9 service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:54.780689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d4f0b-ae23-49d5-b6e4-36c4152d3086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:54.781442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d4f0b-ae23-49d5-b6e4-36c4152d3086 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:54.781906 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d4f0b-ae23-49d5-b6e4-36c4152d3086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:54.782202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d4f0b-ae23-49d5-b6e4-36c4152d3086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:54.784620 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d4f0b-ae23-49d5-b6e4-36c4152d3086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:54.786271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d4f0b-ae23-49d5-b6e4-36c4152d3086 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:54.801187 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 903/1819] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:54.802137 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d4f0b-ae23-49d5-b6e4-36c4152d3086 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:54.814725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d4f0b-ae23-49d5-b6e4-36c4152d3086 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:54.820150 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746cfd64-9a3d-486f-b47c-2110cf3aa596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:54.821054 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cfd64-9a3d-486f-b47c-2110cf3aa596 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:54.821377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cfd64-9a3d-486f-b47c-2110cf3aa596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:54.821844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cfd64-9a3d-486f-b47c-2110cf3aa596 admin admin] PATH_INFO: `/v3/projects/c21dab3cf7bf4e03848ba882ca7f53c8/users/0eb0b50334474fae82659549c510c1f5/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:54.823917 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746cfd64-9a3d-486f-b47c-2110cf3aa596 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c21dab3cf7bf4e03848ba882ca7f53c8, user_id=0eb0b50334474fae82659549c510c1f5, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:54.826772 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746cfd64-9a3d-486f-b47c-2110cf3aa596 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:54.834874 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-39195c37-dbbf-4cda-998c-5c7b3407acab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ba32a55e31d4a3b972c75e35236ce99', 'request_id': 'req-39195c37-dbbf-4cda-998c-5c7b3407acab'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:54.839479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 904/1820] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:54 2022] PUT /identity/v3/projects/c21dab3cf7bf4e03848ba882ca7f53c8/users/0eb0b50334474fae82659549c510c1f5/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:34:54.843700 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-39195c37-dbbf-4cda-998c-5c7b3407acab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ba32a55e31d4a3b972c75e35236ce99', 'request_id': 'req-39195c37-dbbf-4cda-998c-5c7b3407acab'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:54.852095 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-39195c37-dbbf-4cda-998c-5c7b3407acab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ba32a55e31d4a3b972c75e35236ce99', 'request_id': 'req-39195c37-dbbf-4cda-998c-5c7b3407acab'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:54.852739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d35e9-8395-40ac-835c-f430edd03f35 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:54.853047 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d35e9-8395-40ac-835c-f430edd03f35 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:54.853372 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d35e9-8395-40ac-835c-f430edd03f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:54.872794 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-39195c37-dbbf-4cda-998c-5c7b3407acab admin admin] The token cache is being invalidate because project 0ba32a55e31d4a3b972c75e35236ce99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:34:54.873255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-39195c37-dbbf-4cda-998c-5c7b3407acab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:54.874062 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-39195c37-dbbf-4cda-998c-5c7b3407acab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:54.876794 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 917/1821] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:34:54 2022] DELETE /identity/v3/projects/0ba32a55e31d4a3b972c75e35236ce99 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:54.878855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:54.882209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df44f00-6b3f-4107-a8cf-edcdb124068a None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:54.882505 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df44f00-6b3f-4107-a8cf-edcdb124068a None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:54.882791 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df44f00-6b3f-4107-a8cf-edcdb124068a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:54.892727 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-ce4d35e9-8395-40ac-835c-f430edd03f35 None None] MFA Rules not processed for user `0eb0b50334474fae82659549c510c1f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:54.894310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ce4d35e9-8395-40ac-835c-f430edd03f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:54.912909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-2df44f00-6b3f-4107-a8cf-edcdb124068a None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:54.914418 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-2df44f00-6b3f-4107-a8cf-edcdb124068a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:55.021314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 905/1822] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:34:55.027635 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d35e9-8395-40ac-835c-f430edd03f35 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:55.029230 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ce4d35e9-8395-40ac-835c-f430edd03f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:55.032239 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 918/1823] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:55.034986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df44f00-6b3f-4107-a8cf-edcdb124068a None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:55.037341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-2df44f00-6b3f-4107-a8cf-edcdb124068a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:55.109756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df44f00-6b3f-4107-a8cf-edcdb124068a None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:55.114457 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b5a42-cb94-4190-98de-fb0636fbef00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:55.115118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b5a42-cb94-4190-98de-fb0636fbef00 service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:55.115410 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b5a42-cb94-4190-98de-fb0636fbef00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:55.115690 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b5a42-cb94-4190-98de-fb0636fbef00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:55.115998 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d35e9-8395-40ac-835c-f430edd03f35 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:55.120177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bae35-11f4-40b9-a33e-81b29acc1f6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:55.121191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bae35-11f4-40b9-a33e-81b29acc1f6c service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:55.121761 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bae35-11f4-40b9-a33e-81b29acc1f6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:55.122455 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bae35-11f4-40b9-a33e-81b29acc1f6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:55.125512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-704bae35-11f4-40b9-a33e-81b29acc1f6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:55.126619 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b5a42-cb94-4190-98de-fb0636fbef00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:55.128096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b5a42-cb94-4190-98de-fb0636fbef00 service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:55.138960 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 919/1824] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:55.140232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd17f36-efd7-4387-b6cc-7b59f7e93756 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:55.140528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd17f36-efd7-4387-b6cc-7b59f7e93756 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:55.140811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd17f36-efd7-4387-b6cc-7b59f7e93756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:55.166455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-4bd17f36-efd7-4387-b6cc-7b59f7e93756 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:55.168368 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4bd17f36-efd7-4387-b6cc-7b59f7e93756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:55.187127 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 920/1825] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:55.191615 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd17f36-efd7-4387-b6cc-7b59f7e93756 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:55.196375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:55.198762 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704bae35-11f4-40b9-a33e-81b29acc1f6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:55.200517 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704bae35-11f4-40b9-a33e-81b29acc1f6c service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:55.201839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd17f36-efd7-4387-b6cc-7b59f7e93756 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:55.209244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c085499-7b39-4a2a-9ef6-09c7d39678ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:55.209934 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c085499-7b39-4a2a-9ef6-09c7d39678ba admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:55.210559 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c085499-7b39-4a2a-9ef6-09c7d39678ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:55.211099 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c085499-7b39-4a2a-9ef6-09c7d39678ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:55.211889 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 906/1826] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:55 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:34:55.212277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c085499-7b39-4a2a-9ef6-09c7d39678ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:55.212698 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c085499-7b39-4a2a-9ef6-09c7d39678ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:55.214378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c085499-7b39-4a2a-9ef6-09c7d39678ba admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:55.228179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 921/1827] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:34:55.232557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bae35-11f4-40b9-a33e-81b29acc1f6c service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:55.255863 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bae35-11f4-40b9-a33e-81b29acc1f6c service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:55.260676 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad142bb-db0d-4462-a2a6-cfa561673a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:55.261516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad142bb-db0d-4462-a2a6-cfa561673a72 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:55.261870 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad142bb-db0d-4462-a2a6-cfa561673a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:55.262235 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad142bb-db0d-4462-a2a6-cfa561673a72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:55.263018 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad142bb-db0d-4462-a2a6-cfa561673a72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:55.264985 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad142bb-db0d-4462-a2a6-cfa561673a72 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:55.295219 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 907/1828] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:55 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:55.300011 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c085499-7b39-4a2a-9ef6-09c7d39678ba admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:55.309964 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c085499-7b39-4a2a-9ef6-09c7d39678ba admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:55.377500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdbe7c8-676c-4de7-8d25-ae06da5966b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:55.378186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdbe7c8-676c-4de7-8d25-ae06da5966b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:55.378479 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdbe7c8-676c-4de7-8d25-ae06da5966b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:55.378758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdbe7c8-676c-4de7-8d25-ae06da5966b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:55.379558 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdbe7c8-676c-4de7-8d25-ae06da5966b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:55.381790 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdbe7c8-676c-4de7-8d25-ae06da5966b6 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:55.411324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 922/1829] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:55 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:55.415149 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad142bb-db0d-4462-a2a6-cfa561673a72 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:55.426464 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad142bb-db0d-4462-a2a6-cfa561673a72 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:55.432149 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f104b-655a-457a-80f3-0977a1257a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:55.432827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f104b-655a-457a-80f3-0977a1257a30 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:55.433116 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f104b-655a-457a-80f3-0977a1257a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:55.433419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f104b-655a-457a-80f3-0977a1257a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:55.434157 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f104b-655a-457a-80f3-0977a1257a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:55.435743 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f104b-655a-457a-80f3-0977a1257a30 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:55.449760 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 908/1830] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:55.454048 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdbe7c8-676c-4de7-8d25-ae06da5966b6 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:55.466613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdbe7c8-676c-4de7-8d25-ae06da5966b6 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:55.472560 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd89f64-0d67-4f19-905c-94a887d01702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:55.473693 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd89f64-0d67-4f19-905c-94a887d01702 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:55.473693 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd89f64-0d67-4f19-905c-94a887d01702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:55.473901 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd89f64-0d67-4f19-905c-94a887d01702 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:55.474535 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd89f64-0d67-4f19-905c-94a887d01702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:55.476426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd89f64-0d67-4f19-905c-94a887d01702 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:55.490212 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 923/1831] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:55.494312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f104b-655a-457a-80f3-0977a1257a30 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:55.506056 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f104b-655a-457a-80f3-0977a1257a30 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:55.511290 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165aef9-1513-428a-9963-4088df8bd97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:55.512285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165aef9-1513-428a-9963-4088df8bd97f admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:55.512670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165aef9-1513-428a-9963-4088df8bd97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:55.513408 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165aef9-1513-428a-9963-4088df8bd97f admin admin] PATH_INFO: `/v3/projects/b43c41fb210f4ca99ac01be06202f5ee/users/7d34f6535e72410eb1cd234e58cb4141/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:55.531044 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:55.537912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a165aef9-1513-428a-9963-4088df8bd97f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b43c41fb210f4ca99ac01be06202f5ee, user_id=7d34f6535e72410eb1cd234e58cb4141, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:55.542008 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a165aef9-1513-428a-9963-4088df8bd97f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:55.560422 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 909/1832] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:55 2022] PUT /identity/v3/projects/b43c41fb210f4ca99ac01be06202f5ee/users/7d34f6535e72410eb1cd234e58cb4141/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:55.571762 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba169f7-00b8-44f8-9619-2a81f9f2385e None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:55.572208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba169f7-00b8-44f8-9619-2a81f9f2385e None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:55.572569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba169f7-00b8-44f8-9619-2a81f9f2385e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:55.632730 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-9ba169f7-00b8-44f8-9619-2a81f9f2385e None None] MFA Rules not processed for user `7d34f6535e72410eb1cd234e58cb4141`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:55.634691 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-9ba169f7-00b8-44f8-9619-2a81f9f2385e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:55.804825 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165aef9-1513-428a-9963-4088df8bd97f admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:55.806634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a165aef9-1513-428a-9963-4088df8bd97f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:55.812668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 924/1833] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:55.814702 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba169f7-00b8-44f8-9619-2a81f9f2385e None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:55.893910 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165aef9-1513-428a-9963-4088df8bd97f admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:55.898750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:55.899947 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc1ebb1-adff-46ee-aab2-f07d9c891070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:55.900881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1ebb1-adff-46ee-aab2-f07d9c891070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:55.901257 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1ebb1-adff-46ee-aab2-f07d9c891070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:55.901592 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1ebb1-adff-46ee-aab2-f07d9c891070 admin admin] PATH_INFO: `/v3/users/d78bc4d2d3f04f1984d485691a384c3b` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:55.903190 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc1ebb1-adff-46ee-aab2-f07d9c891070 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d78bc4d2d3f04f1984d485691a384c3b)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:55.905481 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc1ebb1-adff-46ee-aab2-f07d9c891070 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:55.905926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba169f7-00b8-44f8-9619-2a81f9f2385e None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:55.919887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f6515-eead-4774-a048-db6e06e6bb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:55.920538 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f6515-eead-4774-a048-db6e06e6bb67 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:55.920833 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f6515-eead-4774-a048-db6e06e6bb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:55.921112 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f6515-eead-4774-a048-db6e06e6bb67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:55.921862 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f6515-eead-4774-a048-db6e06e6bb67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:55.925422 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f6515-eead-4774-a048-db6e06e6bb67 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:55.961928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 925/1834] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:55 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:34:55.965989 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f6515-eead-4774-a048-db6e06e6bb67 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:55.975976 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9dc1ebb1-adff-46ee-aab2-f07d9c891070 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd78bc4d2d3f04f1984d485691a384c3b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:55.988437 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9dc1ebb1-adff-46ee-aab2-f07d9c891070 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd78bc4d2d3f04f1984d485691a384c3b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:55.989185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9dc1ebb1-adff-46ee-aab2-f07d9c891070 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd78bc4d2d3f04f1984d485691a384c3b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:55.996406 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9dc1ebb1-adff-46ee-aab2-f07d9c891070 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd78bc4d2d3f04f1984d485691a384c3b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:56.005995 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-9dc1ebb1-adff-46ee-aab2-f07d9c891070 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd78bc4d2d3f04f1984d485691a384c3b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:56.014467 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 910/1835] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:34:55 2022] DELETE /identity/v3/users/d78bc4d2d3f04f1984d485691a384c3b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:56.019294 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bae52-b8f0-42eb-a0f1-d2e1ab1b304c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:56.019586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bae52-b8f0-42eb-a0f1-d2e1ab1b304c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:56.019875 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bae52-b8f0-42eb-a0f1-d2e1ab1b304c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:56.038471 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-381bae52-b8f0-42eb-a0f1-d2e1ab1b304c None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:56.039695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-381bae52-b8f0-42eb-a0f1-d2e1ab1b304c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:56.061214 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f6515-eead-4774-a048-db6e06e6bb67 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:56.137498 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 911/1836] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:56.151546 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bae52-b8f0-42eb-a0f1-d2e1ab1b304c None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:56.155265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-381bae52-b8f0-42eb-a0f1-d2e1ab1b304c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:56.171458 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10cc73-acd8-4fe9-a810-d51eb2c5df0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:56.172261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10cc73-acd8-4fe9-a810-d51eb2c5df0a admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:56.172677 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10cc73-acd8-4fe9-a810-d51eb2c5df0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:56.172997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10cc73-acd8-4fe9-a810-d51eb2c5df0a admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:56.174129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10cc73-acd8-4fe9-a810-d51eb2c5df0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:56.177480 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10cc73-acd8-4fe9-a810-d51eb2c5df0a admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:56.189840 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:56.287816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bae52-b8f0-42eb-a0f1-d2e1ab1b304c None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:56.288533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 926/1837] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:55 2022] POST /identity/v3/users => generated 293 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:34:56.293524 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10cc73-acd8-4fe9-a810-d51eb2c5df0a admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:56.294949 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a6e41-b8a4-4fb4-ae6a-95055945fbc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:56.295681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a6e41-b8a4-4fb4-ae6a-95055945fbc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:56.295966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a6e41-b8a4-4fb4-ae6a-95055945fbc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:56.296246 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a6e41-b8a4-4fb4-ae6a-95055945fbc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:56.297669 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-fb10cc73-acd8-4fe9-a810-d51eb2c5df0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:56.315462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a6e41-b8a4-4fb4-ae6a-95055945fbc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:56.317586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a6e41-b8a4-4fb4-ae6a-95055945fbc6 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:56.323522 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10cc73-acd8-4fe9-a810-d51eb2c5df0a admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:56.329062 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619ba15-0a60-4468-bddd-c9463e8e291d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:56.329834 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619ba15-0a60-4468-bddd-c9463e8e291d admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:56.330305 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619ba15-0a60-4468-bddd-c9463e8e291d admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:56.330791 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619ba15-0a60-4468-bddd-c9463e8e291d admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:56.331507 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619ba15-0a60-4468-bddd-c9463e8e291d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:56.333231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619ba15-0a60-4468-bddd-c9463e8e291d admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:56.344945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 912/1838] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:56.346219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 927/1839] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:34:56.349845 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a6e41-b8a4-4fb4-ae6a-95055945fbc6 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:56.362756 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a6e41-b8a4-4fb4-ae6a-95055945fbc6 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:56.367813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab8d73-a364-468c-a612-820aa56b4767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:56.368551 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab8d73-a364-468c-a612-820aa56b4767 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:56.368845 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab8d73-a364-468c-a612-820aa56b4767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:56.370027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab8d73-a364-468c-a612-820aa56b4767 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:56.370027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab8d73-a364-468c-a612-820aa56b4767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:56.371464 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab8d73-a364-468c-a612-820aa56b4767 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:56.385270 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 913/1840] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:56.387343 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619ba15-0a60-4468-bddd-c9463e8e291d admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:56.399637 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619ba15-0a60-4468-bddd-c9463e8e291d admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:56.405174 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f414fef-5014-4b64-a68e-84c9f2d012fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:56.406222 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f414fef-5014-4b64-a68e-84c9f2d012fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:56.406565 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f414fef-5014-4b64-a68e-84c9f2d012fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:56.406850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f414fef-5014-4b64-a68e-84c9f2d012fe admin admin] PATH_INFO: `/v3/projects/c7ac7605b76045e6be617934d0246856/users/2782f8e82853499a882696d7df5c4e09/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:56.426576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f414fef-5014-4b64-a68e-84c9f2d012fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7ac7605b76045e6be617934d0246856, user_id=2782f8e82853499a882696d7df5c4e09, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:56.429720 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f414fef-5014-4b64-a68e-84c9f2d012fe admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:56.450057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 928/1841] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:56 2022] PUT /identity/v3/projects/c7ac7605b76045e6be617934d0246856/users/2782f8e82853499a882696d7df5c4e09/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:56.466235 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb194a-fa29-4b5a-b032-208c612b912c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:56.466718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb194a-fa29-4b5a-b032-208c612b912c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:56.467109 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb194a-fa29-4b5a-b032-208c612b912c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:56.513071 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a5cb194a-fa29-4b5a-b032-208c612b912c None None] MFA Rules not processed for user `2782f8e82853499a882696d7df5c4e09`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:56.514806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a5cb194a-fa29-4b5a-b032-208c612b912c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:56.588977 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 914/1842] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:56.593615 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f414fef-5014-4b64-a68e-84c9f2d012fe admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:56.643911 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb194a-fa29-4b5a-b032-208c612b912c None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:56.646972 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a5cb194a-fa29-4b5a-b032-208c612b912c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:56.655752 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:56.692838 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f414fef-5014-4b64-a68e-84c9f2d012fe admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:56.698554 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd1035-685b-43c1-a509-2dcb0e656b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:56.699627 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd1035-685b-43c1-a509-2dcb0e656b73 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:56.700127 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd1035-685b-43c1-a509-2dcb0e656b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:56.701964 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd1035-685b-43c1-a509-2dcb0e656b73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:56.702698 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd1035-685b-43c1-a509-2dcb0e656b73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:56.705324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd1035-685b-43c1-a509-2dcb0e656b73 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:56.725666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb194a-fa29-4b5a-b032-208c612b912c None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:56.730653 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e556bd6-4750-4cdb-a1e9-818858b949dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:56.731477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e556bd6-4750-4cdb-a1e9-818858b949dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:56.731763 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e556bd6-4750-4cdb-a1e9-818858b949dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:56.732045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e556bd6-4750-4cdb-a1e9-818858b949dd admin admin] PATH_INFO: `/v3/projects/5fbd68c034e54a81b0c6dac3aba5ae83` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:56.736988 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 929/1843] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:56 2022] POST /identity/v3/projects => generated 387 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:56.737214 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e556bd6-4750-4cdb-a1e9-818858b949dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fbd68c034e54a81b0c6dac3aba5ae83)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:56.737214 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e556bd6-4750-4cdb-a1e9-818858b949dd admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:56.738895 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd1035-685b-43c1-a509-2dcb0e656b73 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:56.774627 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:56.785999 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5e556bd6-4750-4cdb-a1e9-818858b949dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fbd68c034e54a81b0c6dac3aba5ae83', 'request_id': 'req-5e556bd6-4750-4cdb-a1e9-818858b949dd'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:56.794341 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5e556bd6-4750-4cdb-a1e9-818858b949dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fbd68c034e54a81b0c6dac3aba5ae83', 'request_id': 'req-5e556bd6-4750-4cdb-a1e9-818858b949dd'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:56.805506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd1035-685b-43c1-a509-2dcb0e656b73 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:56.811359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5e556bd6-4750-4cdb-a1e9-818858b949dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fbd68c034e54a81b0c6dac3aba5ae83', 'request_id': 'req-5e556bd6-4750-4cdb-a1e9-818858b949dd'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:56.823361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5e556bd6-4750-4cdb-a1e9-818858b949dd admin admin] The token cache is being invalidate because project 5fbd68c034e54a81b0c6dac3aba5ae83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:34:56.823656 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5e556bd6-4750-4cdb-a1e9-818858b949dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:56.824341 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5e556bd6-4750-4cdb-a1e9-818858b949dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:56.826708 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 915/1844] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:34:56 2022] DELETE /identity/v3/projects/5fbd68c034e54a81b0c6dac3aba5ae83 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:56.831760 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e556bd6-4750-4cdb-a1e9-818858b949dd admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:56.833959 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5e556bd6-4750-4cdb-a1e9-818858b949dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:56.890877 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49a82d-e47b-4867-956f-847403da9494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:56.891945 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49a82d-e47b-4867-956f-847403da9494 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:56.893025 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49a82d-e47b-4867-956f-847403da9494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:56.893629 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49a82d-e47b-4867-956f-847403da9494 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:56.894478 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49a82d-e47b-4867-956f-847403da9494 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:56.898105 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49a82d-e47b-4867-956f-847403da9494 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:56.928018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 930/1845] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:56 2022] POST /identity/v3/users => generated 300 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:34:56.933693 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49a82d-e47b-4867-956f-847403da9494 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:56.936650 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-ce49a82d-e47b-4867-956f-847403da9494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:56.941868 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e556bd6-4750-4cdb-a1e9-818858b949dd admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:56.954123 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49a82d-e47b-4867-956f-847403da9494 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:56.957395 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb98984-8141-41ed-8b9e-5fe83cdbf973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:56.959248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb98984-8141-41ed-8b9e-5fe83cdbf973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:56.959865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb98984-8141-41ed-8b9e-5fe83cdbf973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:56.960295 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb98984-8141-41ed-8b9e-5fe83cdbf973 admin admin] PATH_INFO: `/v3/users/0eb0b50334474fae82659549c510c1f5` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:56.962403 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb98984-8141-41ed-8b9e-5fe83cdbf973 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eb0b50334474fae82659549c510c1f5)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:56.962567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec00f2-4cc6-43b7-99ca-befd2aa11f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:56.963386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec00f2-4cc6-43b7-99ca-befd2aa11f9f admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:56.963982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec00f2-4cc6-43b7-99ca-befd2aa11f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:56.964355 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec00f2-4cc6-43b7-99ca-befd2aa11f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:56.965114 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec00f2-4cc6-43b7-99ca-befd2aa11f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:56.966844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb98984-8141-41ed-8b9e-5fe83cdbf973 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:56.967033 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec00f2-4cc6-43b7-99ca-befd2aa11f9f admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:56.992575 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 931/1846] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:34:56.999307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec00f2-4cc6-43b7-99ca-befd2aa11f9f admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:57.020014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec00f2-4cc6-43b7-99ca-befd2aa11f9f admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:57.030183 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5244320f-131e-4c4a-91ca-0a4ddf41afa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:57.031398 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244320f-131e-4c4a-91ca-0a4ddf41afa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:57.031884 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244320f-131e-4c4a-91ca-0a4ddf41afa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:57.032265 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244320f-131e-4c4a-91ca-0a4ddf41afa0 admin admin] PATH_INFO: `/v3/projects/49b3c62472d449a0a0bc27c532f96912/users/203bc15af47744c38d9e52f83203d801/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:57.052759 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4fb98984-8141-41ed-8b9e-5fe83cdbf973 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eb0b50334474fae82659549c510c1f5'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:57.052996 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5244320f-131e-4c4a-91ca-0a4ddf41afa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49b3c62472d449a0a0bc27c532f96912, user_id=203bc15af47744c38d9e52f83203d801, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:57.055753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5244320f-131e-4c4a-91ca-0a4ddf41afa0 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:57.067544 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4fb98984-8141-41ed-8b9e-5fe83cdbf973 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eb0b50334474fae82659549c510c1f5'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:57.068397 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4fb98984-8141-41ed-8b9e-5fe83cdbf973 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eb0b50334474fae82659549c510c1f5'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:57.071974 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 932/1847] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:56 2022] PUT /identity/v3/projects/49b3c62472d449a0a0bc27c532f96912/users/203bc15af47744c38d9e52f83203d801/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:57.075888 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5244320f-131e-4c4a-91ca-0a4ddf41afa0 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:57.077626 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-5244320f-131e-4c4a-91ca-0a4ddf41afa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:57.079600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4fb98984-8141-41ed-8b9e-5fe83cdbf973 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eb0b50334474fae82659549c510c1f5'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:57.089184 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4fb98984-8141-41ed-8b9e-5fe83cdbf973 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eb0b50334474fae82659549c510c1f5'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:57.097068 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 916/1848] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:34:56 2022] DELETE /identity/v3/users/0eb0b50334474fae82659549c510c1f5 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:57.121477 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:57.170222 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5244320f-131e-4c4a-91ca-0a4ddf41afa0 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:57.246027 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a473a-0e2a-4af9-80d9-1bfd1daa67cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:57.246638 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a473a-0e2a-4af9-80d9-1bfd1daa67cb admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:57.247192 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a473a-0e2a-4af9-80d9-1bfd1daa67cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:57.247469 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a473a-0e2a-4af9-80d9-1bfd1daa67cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:57.248201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a473a-0e2a-4af9-80d9-1bfd1daa67cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:57.249864 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a473a-0e2a-4af9-80d9-1bfd1daa67cb admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:57.264916 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 933/1849] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:57.266145 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb98984-8141-41ed-8b9e-5fe83cdbf973 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:57.278257 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb98984-8141-41ed-8b9e-5fe83cdbf973 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:57.283528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb7450-0b61-4471-b5d3-22f10c8f053d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:57.284430 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb7450-0b61-4471-b5d3-22f10c8f053d admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:57.284722 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb7450-0b61-4471-b5d3-22f10c8f053d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:57.285002 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb7450-0b61-4471-b5d3-22f10c8f053d admin admin] PATH_INFO: `/v3/projects/49b3c62472d449a0a0bc27c532f96912/users/203bc15af47744c38d9e52f83203d801/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:57.287095 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fb7450-0b61-4471-b5d3-22f10c8f053d admin admin] RBAC: Authorizing `identity:create_grant(project_id=49b3c62472d449a0a0bc27c532f96912, user_id=203bc15af47744c38d9e52f83203d801, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:57.289872 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fb7450-0b61-4471-b5d3-22f10c8f053d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:57.300631 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 917/1850] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:57 2022] PUT /identity/v3/projects/49b3c62472d449a0a0bc27c532f96912/users/203bc15af47744c38d9e52f83203d801/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:34:57.310824 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd029b81-82f1-4ea9-a621-c2f7881bee54 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:57.311230 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd029b81-82f1-4ea9-a621-c2f7881bee54 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:57.311522 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd029b81-82f1-4ea9-a621-c2f7881bee54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:57.340030 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-bd029b81-82f1-4ea9-a621-c2f7881bee54 None None] MFA Rules not processed for user `203bc15af47744c38d9e52f83203d801`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:57.341261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-bd029b81-82f1-4ea9-a621-c2f7881bee54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:57.398431 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb7450-0b61-4471-b5d3-22f10c8f053d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:57.400148 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-64fb7450-0b61-4471-b5d3-22f10c8f053d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:57.439500 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:57.462809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 934/1851] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:57.473960 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd029b81-82f1-4ea9-a621-c2f7881bee54 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:57.475715 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-bd029b81-82f1-4ea9-a621-c2f7881bee54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:57.515332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb7450-0b61-4471-b5d3-22f10c8f053d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:57.538209 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467800c-fb3c-4774-830e-402fade29d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:57.539124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467800c-fb3c-4774-830e-402fade29d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:57.539410 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467800c-fb3c-4774-830e-402fade29d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:57.539739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467800c-fb3c-4774-830e-402fade29d49 admin admin] PATH_INFO: `/v3/projects/c21dab3cf7bf4e03848ba882ca7f53c8` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:57.541042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467800c-fb3c-4774-830e-402fade29d49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c21dab3cf7bf4e03848ba882ca7f53c8)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:57.543028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467800c-fb3c-4774-830e-402fade29d49 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:57.619197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd029b81-82f1-4ea9-a621-c2f7881bee54 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:57.657751 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:57.658484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d467800c-fb3c-4774-830e-402fade29d49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c21dab3cf7bf4e03848ba882ca7f53c8', 'request_id': 'req-d467800c-fb3c-4774-830e-402fade29d49'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:57.678698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d467800c-fb3c-4774-830e-402fade29d49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c21dab3cf7bf4e03848ba882ca7f53c8', 'request_id': 'req-d467800c-fb3c-4774-830e-402fade29d49'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:57.695323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d467800c-fb3c-4774-830e-402fade29d49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c21dab3cf7bf4e03848ba882ca7f53c8', 'request_id': 'req-d467800c-fb3c-4774-830e-402fade29d49'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:57.722698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d467800c-fb3c-4774-830e-402fade29d49 admin admin] The token cache is being invalidate because project c21dab3cf7bf4e03848ba882ca7f53c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:34:57.722994 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d467800c-fb3c-4774-830e-402fade29d49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:57.723796 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d467800c-fb3c-4774-830e-402fade29d49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:57.729108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 918/1852] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:34:57 2022] DELETE /identity/v3/projects/c21dab3cf7bf4e03848ba882ca7f53c8 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:57.734562 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254a8c2-bbe7-48de-b93c-1c80cbdcfe7d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:57.734856 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254a8c2-bbe7-48de-b93c-1c80cbdcfe7d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:57.735138 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254a8c2-bbe7-48de-b93c-1c80cbdcfe7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:57.757063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8654a-b999-46d5-9586-7856ab842c0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:57.757705 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8654a-b999-46d5-9586-7856ab842c0c service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:57.757992 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8654a-b999-46d5-9586-7856ab842c0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:57.758276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8654a-b999-46d5-9586-7856ab842c0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:57.767518 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-5ca8654a-b999-46d5-9586-7856ab842c0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:57.795654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-6254a8c2-bbe7-48de-b93c-1c80cbdcfe7d None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:57.797107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6254a8c2-bbe7-48de-b93c-1c80cbdcfe7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:57.880100 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca8654a-b999-46d5-9586-7856ab842c0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:57.882636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca8654a-b999-46d5-9586-7856ab842c0c service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:57.896020 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 935/1853] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:57.921456 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 919/1854] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:57.930903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee1ad3-e68f-468c-8592-239d62203c21 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:57.931208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee1ad3-e68f-468c-8592-239d62203c21 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:57.931984 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee1ad3-e68f-468c-8592-239d62203c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:57.959837 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-86ee1ad3-e68f-468c-8592-239d62203c21 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:57.963350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-86ee1ad3-e68f-468c-8592-239d62203c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:57.982047 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 936/1855] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:57.982980 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254a8c2-bbe7-48de-b93c-1c80cbdcfe7d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:57.997923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254a8c2-bbe7-48de-b93c-1c80cbdcfe7d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:58.008605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e68ca-b422-4899-a9eb-378868a3b27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:58.009400 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e68ca-b422-4899-a9eb-378868a3b27c admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:58.009693 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e68ca-b422-4899-a9eb-378868a3b27c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:58.009976 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e68ca-b422-4899-a9eb-378868a3b27c admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:58.010729 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e68ca-b422-4899-a9eb-378868a3b27c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:58.011045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e68ca-b422-4899-a9eb-378868a3b27c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:58.012680 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e68ca-b422-4899-a9eb-378868a3b27c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:58.036659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 920/1856] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:58.047647 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee1ad3-e68f-468c-8592-239d62203c21 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:58.075768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee1ad3-e68f-468c-8592-239d62203c21 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:58.087141 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615db09e-2bfb-4a6c-991b-f71ffbea5f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:58.087839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615db09e-2bfb-4a6c-991b-f71ffbea5f60 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:58.088128 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615db09e-2bfb-4a6c-991b-f71ffbea5f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:58.088814 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615db09e-2bfb-4a6c-991b-f71ffbea5f60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:58.089601 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615db09e-2bfb-4a6c-991b-f71ffbea5f60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:58.092094 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615db09e-2bfb-4a6c-991b-f71ffbea5f60 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:58.134106 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 937/1857] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:58 2022] POST /identity/v3/projects => generated 371 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:58.138249 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e68ca-b422-4899-a9eb-378868a3b27c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:58.151882 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e68ca-b422-4899-a9eb-378868a3b27c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:58.227517 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:58.257858 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1adff-ecb1-4cd7-9657-61c3111463e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:58.258777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1adff-ecb1-4cd7-9657-61c3111463e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:58.259104 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1adff-ecb1-4cd7-9657-61c3111463e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:58.259535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1adff-ecb1-4cd7-9657-61c3111463e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:58.260402 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1adff-ecb1-4cd7-9657-61c3111463e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:58.262689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1adff-ecb1-4cd7-9657-61c3111463e4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:58.290111 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 921/1858] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:58 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:58.294140 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615db09e-2bfb-4a6c-991b-f71ffbea5f60 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:58.309218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615db09e-2bfb-4a6c-991b-f71ffbea5f60 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:58.314884 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363fda4-f1ac-45e7-be00-461fa2851d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:58.315765 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363fda4-f1ac-45e7-be00-461fa2851d84 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:58.316192 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363fda4-f1ac-45e7-be00-461fa2851d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:58.316608 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363fda4-f1ac-45e7-be00-461fa2851d84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:58.317556 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4363fda4-f1ac-45e7-be00-461fa2851d84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:58.319532 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4363fda4-f1ac-45e7-be00-461fa2851d84 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:58.331007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 938/1859] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:58.335371 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1adff-ecb1-4cd7-9657-61c3111463e4 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:58.346218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1adff-ecb1-4cd7-9657-61c3111463e4 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:58.351255 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab3969-c93b-4704-8c49-78c76d41b4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:58.351940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab3969-c93b-4704-8c49-78c76d41b4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:58.352234 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab3969-c93b-4704-8c49-78c76d41b4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:58.352520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab3969-c93b-4704-8c49-78c76d41b4b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:58.353296 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab3969-c93b-4704-8c49-78c76d41b4b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:58.354967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab3969-c93b-4704-8c49-78c76d41b4b0 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:58.366418 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 922/1860] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:34:58.371208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363fda4-f1ac-45e7-be00-461fa2851d84 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:58.382736 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363fda4-f1ac-45e7-be00-461fa2851d84 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:58.387453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7dab1-d207-4833-b47d-dc2361039177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:58.388350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7dab1-d207-4833-b47d-dc2361039177 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:58.388645 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7dab1-d207-4833-b47d-dc2361039177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:58.388968 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7dab1-d207-4833-b47d-dc2361039177 admin admin] PATH_INFO: `/v3/projects/c15148d9bba74dd0bb4cc6cc8960f1d6/users/c1ab33725f19417ebf15df836bf74ab9/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:58.404793 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7dab1-d207-4833-b47d-dc2361039177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c15148d9bba74dd0bb4cc6cc8960f1d6, user_id=c1ab33725f19417ebf15df836bf74ab9, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:58.407541 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7dab1-d207-4833-b47d-dc2361039177 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:58.421790 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 939/1861] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:58 2022] PUT /identity/v3/projects/c15148d9bba74dd0bb4cc6cc8960f1d6/users/c1ab33725f19417ebf15df836bf74ab9/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:58.427683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67326275-3e7d-48eb-8b46-793e7f1027ac None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:58.427980 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67326275-3e7d-48eb-8b46-793e7f1027ac None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:58.428276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67326275-3e7d-48eb-8b46-793e7f1027ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:58.446484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-67326275-3e7d-48eb-8b46-793e7f1027ac None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:58.447688 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-67326275-3e7d-48eb-8b46-793e7f1027ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:58.511296 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 923/1862] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:34:58.521226 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7dab1-d207-4833-b47d-dc2361039177 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:58.522976 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-60a7dab1-d207-4833-b47d-dc2361039177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:58.582186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7dab1-d207-4833-b47d-dc2361039177 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:58.585999 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087db43-63ff-40dd-89c0-5a48e3954e4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:58.586656 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087db43-63ff-40dd-89c0-5a48e3954e4d service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:58.586954 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087db43-63ff-40dd-89c0-5a48e3954e4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:58.587234 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087db43-63ff-40dd-89c0-5a48e3954e4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:58.599818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087db43-63ff-40dd-89c0-5a48e3954e4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:58.599818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087db43-63ff-40dd-89c0-5a48e3954e4d service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:58.607669 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67326275-3e7d-48eb-8b46-793e7f1027ac None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:58.608478 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 940/1863] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:58.609263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-67326275-3e7d-48eb-8b46-793e7f1027ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:58.675479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67326275-3e7d-48eb-8b46-793e7f1027ac None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:58.680095 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57159fc2-952b-4522-8c63-5ed0296e610a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:58.680948 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57159fc2-952b-4522-8c63-5ed0296e610a service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:58.681335 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57159fc2-952b-4522-8c63-5ed0296e610a service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:58.681593 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57159fc2-952b-4522-8c63-5ed0296e610a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:58.683727 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-57159fc2-952b-4522-8c63-5ed0296e610a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:58.752811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57159fc2-952b-4522-8c63-5ed0296e610a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:58.754669 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57159fc2-952b-4522-8c63-5ed0296e610a service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:58.766950 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 924/1864] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:34:59.512952 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541ac32-9abd-46e4-8c5f-0bc67b2b5d4a None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:59.514073 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541ac32-9abd-46e4-8c5f-0bc67b2b5d4a None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:59.514574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541ac32-9abd-46e4-8c5f-0bc67b2b5d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:59.525244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:59.551465 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-a541ac32-9abd-46e4-8c5f-0bc67b2b5d4a None None] MFA Rules not processed for user `c1ab33725f19417ebf15df836bf74ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:59.553021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-a541ac32-9abd-46e4-8c5f-0bc67b2b5d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:59.643125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 941/1865] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:34:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:59.643999 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57159fc2-952b-4522-8c63-5ed0296e610a service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:59.655357 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57159fc2-952b-4522-8c63-5ed0296e610a service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:59.660386 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fefee9-4e12-480e-9846-f3d74ee844ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:59.661068 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fefee9-4e12-480e-9846-f3d74ee844ea admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:59.661412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fefee9-4e12-480e-9846-f3d74ee844ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:59.661808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fefee9-4e12-480e-9846-f3d74ee844ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:59.662518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fefee9-4e12-480e-9846-f3d74ee844ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:59.664422 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fefee9-4e12-480e-9846-f3d74ee844ea admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:59.683431 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:59.693185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 925/1866] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:34:59 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:59.696945 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541ac32-9abd-46e4-8c5f-0bc67b2b5d4a None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:59.708311 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541ac32-9abd-46e4-8c5f-0bc67b2b5d4a None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:59.764580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bd03d-4f7f-4348-9645-d41dfb548a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:59.765300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bd03d-4f7f-4348-9645-d41dfb548a09 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:59.765600 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bd03d-4f7f-4348-9645-d41dfb548a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:59.765887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bd03d-4f7f-4348-9645-d41dfb548a09 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:59.766592 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877bd03d-4f7f-4348-9645-d41dfb548a09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:59.769709 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877bd03d-4f7f-4348-9645-d41dfb548a09 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:59.794159 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 942/1867] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:34:59 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:59.795270 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fefee9-4e12-480e-9846-f3d74ee844ea admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:59.806660 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fefee9-4e12-480e-9846-f3d74ee844ea admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:59.811983 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b26951-fc5e-4c86-80cb-ebca3788c7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:59.812659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b26951-fc5e-4c86-80cb-ebca3788c7e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:59.812945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b26951-fc5e-4c86-80cb-ebca3788c7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:59.813266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b26951-fc5e-4c86-80cb-ebca3788c7e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:59.814164 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b26951-fc5e-4c86-80cb-ebca3788c7e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:59.815760 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b26951-fc5e-4c86-80cb-ebca3788c7e6 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:59.826291 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 926/1868] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:34:59.831144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bd03d-4f7f-4348-9645-d41dfb548a09 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:59.841035 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bd03d-4f7f-4348-9645-d41dfb548a09 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:59.846297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311301f-0477-47e9-9e53-4bdc5c39faf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:59.847302 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311301f-0477-47e9-9e53-4bdc5c39faf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:59.847600 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311301f-0477-47e9-9e53-4bdc5c39faf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:59.847882 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311301f-0477-47e9-9e53-4bdc5c39faf8 admin admin] PATH_INFO: `/v3/projects/2c4694d1b70a48608cffb580deef1741/users/7b72d74533104027958828ccda6af289/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:59.864991 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311301f-0477-47e9-9e53-4bdc5c39faf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c4694d1b70a48608cffb580deef1741, user_id=7b72d74533104027958828ccda6af289, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:59.867783 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311301f-0477-47e9-9e53-4bdc5c39faf8 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:59.882172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 943/1869] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:59 2022] PUT /identity/v3/projects/2c4694d1b70a48608cffb580deef1741/users/7b72d74533104027958828ccda6af289/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:59.885996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b26951-fc5e-4c86-80cb-ebca3788c7e6 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:59.897751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b26951-fc5e-4c86-80cb-ebca3788c7e6 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:59.958848 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df440b6-80cb-4c70-a039-cc322edbe8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:59.959536 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df440b6-80cb-4c70-a039-cc322edbe8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:59.959829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df440b6-80cb-4c70-a039-cc322edbe8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:59.960118 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df440b6-80cb-4c70-a039-cc322edbe8e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:59.960852 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df440b6-80cb-4c70-a039-cc322edbe8e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:59.962495 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df440b6-80cb-4c70-a039-cc322edbe8e5 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:59.972970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 927/1870] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:34:59.977394 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311301f-0477-47e9-9e53-4bdc5c39faf8 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:59.989555 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311301f-0477-47e9-9e53-4bdc5c39faf8 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:59.994233 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b8e12-fb8a-482d-a37a-805de3c1b830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:59.995145 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b8e12-fb8a-482d-a37a-805de3c1b830 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:59.995437 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b8e12-fb8a-482d-a37a-805de3c1b830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:59.995722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b8e12-fb8a-482d-a37a-805de3c1b830 admin admin] PATH_INFO: `/v3/projects/2c4694d1b70a48608cffb580deef1741/users/7b72d74533104027958828ccda6af289/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:59.998052 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b8e12-fb8a-482d-a37a-805de3c1b830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c4694d1b70a48608cffb580deef1741, user_id=7b72d74533104027958828ccda6af289, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:00.000697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b8e12-fb8a-482d-a37a-805de3c1b830 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:00.011659 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 944/1871] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:34:59 2022] PUT /identity/v3/projects/2c4694d1b70a48608cffb580deef1741/users/7b72d74533104027958828ccda6af289/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:00.687325 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732c071-3226-4647-8cce-2ef2dcdc3db7 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:00.687905 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732c071-3226-4647-8cce-2ef2dcdc3db7 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:00.687905 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732c071-3226-4647-8cce-2ef2dcdc3db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:00.719797 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-f732c071-3226-4647-8cce-2ef2dcdc3db7 None None] MFA Rules not processed for user `7b72d74533104027958828ccda6af289`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:00.721374 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f732c071-3226-4647-8cce-2ef2dcdc3db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:00.830924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 928/1872] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:00.838602 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b8e12-fb8a-482d-a37a-805de3c1b830 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:00.904547 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:00.919071 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b8e12-fb8a-482d-a37a-805de3c1b830 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:00.933767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754810d-9706-47c9-9258-7354d97c8fb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:00.935557 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754810d-9706-47c9-9258-7354d97c8fb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:00.936036 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754810d-9706-47c9-9258-7354d97c8fb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:00.936482 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754810d-9706-47c9-9258-7354d97c8fb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:01.037798 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e754810d-9706-47c9-9258-7354d97c8fb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:01.039704 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e754810d-9706-47c9-9258-7354d97c8fb8 service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:01.061939 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 945/1873] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:04.797535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f732c071-3226-4647-8cce-2ef2dcdc3db7 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:04.812794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f732c071-3226-4647-8cce-2ef2dcdc3db7 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:04.818008 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e73e6-4ffa-44e8-909e-1ed40c7445e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:04.818695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e73e6-4ffa-44e8-909e-1ed40c7445e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:04.819353 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e73e6-4ffa-44e8-909e-1ed40c7445e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:04.819657 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e73e6-4ffa-44e8-909e-1ed40c7445e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:04.848885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e73e6-4ffa-44e8-909e-1ed40c7445e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:04.850507 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e73e6-4ffa-44e8-909e-1ed40c7445e7 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:04.863333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc05b6-249e-4742-93a2-2de1447df779 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:04.864323 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc05b6-249e-4742-93a2-2de1447df779 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:04.864803 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc05b6-249e-4742-93a2-2de1447df779 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:04.865523 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 929/1874] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:04 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:04.866907 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 946/1875] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 17:35:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:35:04.883813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92a277-a6ac-4fdf-8cda-9113ad8b5788 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:04.884253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92a277-a6ac-4fdf-8cda-9113ad8b5788 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:04.884866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92a277-a6ac-4fdf-8cda-9113ad8b5788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:04.916873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-ab92a277-a6ac-4fdf-8cda-9113ad8b5788 None None] MFA Rules not processed for user `689a31e26c6b454a91c3fd79678b4d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:04.919854 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-ab92a277-a6ac-4fdf-8cda-9113ad8b5788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:05.072856 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 947/1876] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 17:35:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:05.094070 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e73e6-4ffa-44e8-909e-1ed40c7445e7 service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:05.096134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-cd7e73e6-4ffa-44e8-909e-1ed40c7445e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:05.112742 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e73e6-4ffa-44e8-909e-1ed40c7445e7 service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:05.118532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565a9fc-e07c-4344-b6f7-521c3dc74b29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:05.119202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565a9fc-e07c-4344-b6f7-521c3dc74b29 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:05.119496 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565a9fc-e07c-4344-b6f7-521c3dc74b29 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:05.119779 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565a9fc-e07c-4344-b6f7-521c3dc74b29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:05.130696 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565a9fc-e07c-4344-b6f7-521c3dc74b29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:05.132519 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565a9fc-e07c-4344-b6f7-521c3dc74b29 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:05.144810 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 930/1877] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:07.365593 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92a277-a6ac-4fdf-8cda-9113ad8b5788 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:07.367691 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-ab92a277-a6ac-4fdf-8cda-9113ad8b5788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:07.384073 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92a277-a6ac-4fdf-8cda-9113ad8b5788 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:07.387654 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299c5f7-6a94-4775-a9d2-f1ff59ba0fcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:07.388342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299c5f7-6a94-4775-a9d2-f1ff59ba0fcb service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:07.388662 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299c5f7-6a94-4775-a9d2-f1ff59ba0fcb service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:07.388978 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299c5f7-6a94-4775-a9d2-f1ff59ba0fcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:07.393388 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4299c5f7-6a94-4775-a9d2-f1ff59ba0fcb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:07.460131 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299c5f7-6a94-4775-a9d2-f1ff59ba0fcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:07.461810 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299c5f7-6a94-4775-a9d2-f1ff59ba0fcb service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:07.474237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 948/1878] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:07.837102 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565a9fc-e07c-4344-b6f7-521c3dc74b29 service nova] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:07.838779 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c565a9fc-e07c-4344-b6f7-521c3dc74b29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:07.923752 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:07.971643 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565a9fc-e07c-4344-b6f7-521c3dc74b29 service nova] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:07.988108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229540ce-099c-4c98-9b43-b2d2910b511b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:07.989022 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229540ce-099c-4c98-9b43-b2d2910b511b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:07.989347 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229540ce-099c-4c98-9b43-b2d2910b511b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:07.989631 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229540ce-099c-4c98-9b43-b2d2910b511b admin admin] PATH_INFO: `/v3/users/0c678bcb17de4c65ba373d327edde6bc` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:07.991564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229540ce-099c-4c98-9b43-b2d2910b511b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c678bcb17de4c65ba373d327edde6bc)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:07.993588 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229540ce-099c-4c98-9b43-b2d2910b511b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:08.059480 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-229540ce-099c-4c98-9b43-b2d2910b511b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c678bcb17de4c65ba373d327edde6bc'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:08.083218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-229540ce-099c-4c98-9b43-b2d2910b511b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c678bcb17de4c65ba373d327edde6bc'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:08.084036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-229540ce-099c-4c98-9b43-b2d2910b511b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c678bcb17de4c65ba373d327edde6bc'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:08.101492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-229540ce-099c-4c98-9b43-b2d2910b511b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c678bcb17de4c65ba373d327edde6bc'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:08.115105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-229540ce-099c-4c98-9b43-b2d2910b511b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c678bcb17de4c65ba373d327edde6bc'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:08.125845 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 931/1879] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:07 2022] DELETE /identity/v3/users/0c678bcb17de4c65ba373d327edde6bc => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:08.130907 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72188af-a278-45d6-8a20-1c426efd0469 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:08.131140 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72188af-a278-45d6-8a20-1c426efd0469 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:08.131465 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72188af-a278-45d6-8a20-1c426efd0469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:08.152531 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-c72188af-a278-45d6-8a20-1c426efd0469 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:08.155221 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-c72188af-a278-45d6-8a20-1c426efd0469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:08.217243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:08.276334 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 949/1880] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:08.288338 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229540ce-099c-4c98-9b43-b2d2910b511b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:08.290446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-229540ce-099c-4c98-9b43-b2d2910b511b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:08.405763 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229540ce-099c-4c98-9b43-b2d2910b511b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:08.410509 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b6ed1-ea28-4bc7-9639-6208cfc6dbcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:08.411359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b6ed1-ea28-4bc7-9639-6208cfc6dbcd service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:08.411653 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b6ed1-ea28-4bc7-9639-6208cfc6dbcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:08.412700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b6ed1-ea28-4bc7-9639-6208cfc6dbcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:08.444804 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251b6ed1-ea28-4bc7-9639-6208cfc6dbcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:08.447382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251b6ed1-ea28-4bc7-9639-6208cfc6dbcd service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:08.472755 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 932/1881] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:09.006308 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72188af-a278-45d6-8a20-1c426efd0469 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:09.007992 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-c72188af-a278-45d6-8a20-1c426efd0469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:09.022847 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72188af-a278-45d6-8a20-1c426efd0469 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:09.028393 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e44d3-9e65-42f3-b94c-7223cf353cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:09.029325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e44d3-9e65-42f3-b94c-7223cf353cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:09.029620 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e44d3-9e65-42f3-b94c-7223cf353cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:09.030088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e44d3-9e65-42f3-b94c-7223cf353cb6 admin admin] PATH_INFO: `/v3/projects/64a0889577c9418ab0af5a10ccc89e79` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:09.031343 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e44d3-9e65-42f3-b94c-7223cf353cb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64a0889577c9418ab0af5a10ccc89e79)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:09.034946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e44d3-9e65-42f3-b94c-7223cf353cb6 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:09.098036 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b24e44d3-9e65-42f3-b94c-7223cf353cb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64a0889577c9418ab0af5a10ccc89e79', 'request_id': 'req-b24e44d3-9e65-42f3-b94c-7223cf353cb6'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:09.103840 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b24e44d3-9e65-42f3-b94c-7223cf353cb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64a0889577c9418ab0af5a10ccc89e79', 'request_id': 'req-b24e44d3-9e65-42f3-b94c-7223cf353cb6'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:09.111742 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b24e44d3-9e65-42f3-b94c-7223cf353cb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64a0889577c9418ab0af5a10ccc89e79', 'request_id': 'req-b24e44d3-9e65-42f3-b94c-7223cf353cb6'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:09.125693 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b24e44d3-9e65-42f3-b94c-7223cf353cb6 admin admin] The token cache is being invalidate because project 64a0889577c9418ab0af5a10ccc89e79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:09.127777 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b24e44d3-9e65-42f3-b94c-7223cf353cb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:09.128462 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b24e44d3-9e65-42f3-b94c-7223cf353cb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:09.130413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 950/1882] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:09 2022] DELETE /identity/v3/projects/64a0889577c9418ab0af5a10ccc89e79 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:09.135190 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b6ed1-ea28-4bc7-9639-6208cfc6dbcd service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:09.145977 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b6ed1-ea28-4bc7-9639-6208cfc6dbcd service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:09.225604 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bf511-7056-4bc0-9a91-a8a78fa763f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:09.226512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bf511-7056-4bc0-9a91-a8a78fa763f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:09.226802 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bf511-7056-4bc0-9a91-a8a78fa763f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:09.227087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bf511-7056-4bc0-9a91-a8a78fa763f0 admin admin] PATH_INFO: `/v3/users/ce19cc3b78d14dcb8b6cc27a9a1caf3e` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:09.228339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bf511-7056-4bc0-9a91-a8a78fa763f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce19cc3b78d14dcb8b6cc27a9a1caf3e)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:09.235244 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bf511-7056-4bc0-9a91-a8a78fa763f0 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:09.236185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:09.280065 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cc5bf511-7056-4bc0-9a91-a8a78fa763f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce19cc3b78d14dcb8b6cc27a9a1caf3e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:09.293073 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cc5bf511-7056-4bc0-9a91-a8a78fa763f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce19cc3b78d14dcb8b6cc27a9a1caf3e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:09.293805 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cc5bf511-7056-4bc0-9a91-a8a78fa763f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce19cc3b78d14dcb8b6cc27a9a1caf3e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:09.301991 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cc5bf511-7056-4bc0-9a91-a8a78fa763f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce19cc3b78d14dcb8b6cc27a9a1caf3e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:09.309831 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cc5bf511-7056-4bc0-9a91-a8a78fa763f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce19cc3b78d14dcb8b6cc27a9a1caf3e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:09.316210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 933/1883] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:09 2022] DELETE /identity/v3/users/ce19cc3b78d14dcb8b6cc27a9a1caf3e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:09.587752 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e44d3-9e65-42f3-b94c-7223cf353cb6 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:09.589338 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b24e44d3-9e65-42f3-b94c-7223cf353cb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:09.604502 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e44d3-9e65-42f3-b94c-7223cf353cb6 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:09.687332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c17d4b-69bd-40c6-aa39-68c8dd88f32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:09.688559 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c17d4b-69bd-40c6-aa39-68c8dd88f32e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:09.689646 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c17d4b-69bd-40c6-aa39-68c8dd88f32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:09.689944 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c17d4b-69bd-40c6-aa39-68c8dd88f32e admin admin] PATH_INFO: `/v3/projects/fa2168e20e0d4084b17b331d1e1b019a` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:09.691351 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c17d4b-69bd-40c6-aa39-68c8dd88f32e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa2168e20e0d4084b17b331d1e1b019a)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:09.693331 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c17d4b-69bd-40c6-aa39-68c8dd88f32e admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:09.838353 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b7c17d4b-69bd-40c6-aa39-68c8dd88f32e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa2168e20e0d4084b17b331d1e1b019a', 'request_id': 'req-b7c17d4b-69bd-40c6-aa39-68c8dd88f32e'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:09.859464 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:09.864553 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b7c17d4b-69bd-40c6-aa39-68c8dd88f32e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa2168e20e0d4084b17b331d1e1b019a', 'request_id': 'req-b7c17d4b-69bd-40c6-aa39-68c8dd88f32e'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:09.886168 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b7c17d4b-69bd-40c6-aa39-68c8dd88f32e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa2168e20e0d4084b17b331d1e1b019a', 'request_id': 'req-b7c17d4b-69bd-40c6-aa39-68c8dd88f32e'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:09.916499 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b7c17d4b-69bd-40c6-aa39-68c8dd88f32e admin admin] The token cache is being invalidate because project fa2168e20e0d4084b17b331d1e1b019a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:09.916808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b7c17d4b-69bd-40c6-aa39-68c8dd88f32e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:09.918561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b7c17d4b-69bd-40c6-aa39-68c8dd88f32e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:09.925976 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 951/1884] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:09 2022] DELETE /identity/v3/projects/fa2168e20e0d4084b17b331d1e1b019a => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:09.933453 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f27ace-3138-48d5-b580-315feb066332 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:09.933754 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f27ace-3138-48d5-b580-315feb066332 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:09.934083 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f27ace-3138-48d5-b580-315feb066332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:09.992644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a3f27ace-3138-48d5-b580-315feb066332 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:09.994279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a3f27ace-3138-48d5-b580-315feb066332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:10.188035 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 934/1885] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:10.190874 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af055308-29ea-4dca-bd0d-93c8eb3066d4 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:10.191314 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af055308-29ea-4dca-bd0d-93c8eb3066d4 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:10.191456 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af055308-29ea-4dca-bd0d-93c8eb3066d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:10.229726 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-af055308-29ea-4dca-bd0d-93c8eb3066d4 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:10.231149 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-af055308-29ea-4dca-bd0d-93c8eb3066d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:10.246986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 952/1886] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:10.250740 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f27ace-3138-48d5-b580-315feb066332 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:10.264528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f27ace-3138-48d5-b580-315feb066332 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:10.280256 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a4e04-cb98-4a6d-9de7-6ab0a3576daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:10.281122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a4e04-cb98-4a6d-9de7-6ab0a3576daa admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:10.281447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a4e04-cb98-4a6d-9de7-6ab0a3576daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:10.281735 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a4e04-cb98-4a6d-9de7-6ab0a3576daa admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:10.283120 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a4e04-cb98-4a6d-9de7-6ab0a3576daa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:35:10.283441 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a4e04-cb98-4a6d-9de7-6ab0a3576daa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:10.285534 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a4e04-cb98-4a6d-9de7-6ab0a3576daa admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:10.301264 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 935/1887] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:35:10.305043 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af055308-29ea-4dca-bd0d-93c8eb3066d4 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:10.317801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af055308-29ea-4dca-bd0d-93c8eb3066d4 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:10.324040 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4599a2-1c5c-4bb7-a30f-4a8152b6070e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:10.324748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4599a2-1c5c-4bb7-a30f-4a8152b6070e admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:10.325097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4599a2-1c5c-4bb7-a30f-4a8152b6070e admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:10.325421 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4599a2-1c5c-4bb7-a30f-4a8152b6070e admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:10.326261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4599a2-1c5c-4bb7-a30f-4a8152b6070e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:10.328249 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4599a2-1c5c-4bb7-a30f-4a8152b6070e admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:10.358292 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 953/1888] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:10 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:35:10.362632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a4e04-cb98-4a6d-9de7-6ab0a3576daa admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:10.375611 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a4e04-cb98-4a6d-9de7-6ab0a3576daa admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:10.442230 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8a8e6-83bf-4553-8c86-9ceaebcea3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:10.443794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8a8e6-83bf-4553-8c86-9ceaebcea3a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:10.444240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8a8e6-83bf-4553-8c86-9ceaebcea3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:10.444679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8a8e6-83bf-4553-8c86-9ceaebcea3a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:10.445473 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8a8e6-83bf-4553-8c86-9ceaebcea3a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:10.447366 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8a8e6-83bf-4553-8c86-9ceaebcea3a0 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:10.467924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:10.476906 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 936/1889] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:10 2022] POST /identity/v3/users => generated 284 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:10.480384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4599a2-1c5c-4bb7-a30f-4a8152b6070e admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:10.496222 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4599a2-1c5c-4bb7-a30f-4a8152b6070e admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:10.509454 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040836f-44d0-4cb4-bf2d-6880f9aea0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:10.510359 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040836f-44d0-4cb4-bf2d-6880f9aea0fd admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:10.510873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040836f-44d0-4cb4-bf2d-6880f9aea0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:10.511671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040836f-44d0-4cb4-bf2d-6880f9aea0fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:10.512947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040836f-44d0-4cb4-bf2d-6880f9aea0fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:10.514867 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040836f-44d0-4cb4-bf2d-6880f9aea0fd admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:10.534736 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 954/1890] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:10.535920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8a8e6-83bf-4553-8c86-9ceaebcea3a0 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:10.549253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8a8e6-83bf-4553-8c86-9ceaebcea3a0 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:10.559052 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf446420-9c99-4893-9665-ed9cba2d640c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:10.559998 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf446420-9c99-4893-9665-ed9cba2d640c admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:10.560454 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf446420-9c99-4893-9665-ed9cba2d640c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:10.560880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf446420-9c99-4893-9665-ed9cba2d640c admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:10.562096 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf446420-9c99-4893-9665-ed9cba2d640c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:10.565415 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf446420-9c99-4893-9665-ed9cba2d640c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:10.580624 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 937/1891] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:35:10.584800 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040836f-44d0-4cb4-bf2d-6880f9aea0fd admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:10.599939 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040836f-44d0-4cb4-bf2d-6880f9aea0fd admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:10.615347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5ea37-d44b-473c-870b-236a9e968300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:10.617351 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5ea37-d44b-473c-870b-236a9e968300 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:10.617985 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5ea37-d44b-473c-870b-236a9e968300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:10.618476 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5ea37-d44b-473c-870b-236a9e968300 admin admin] PATH_INFO: `/v3/projects/20bf4fe4ecc74735a2abdb5068b0785a/users/305767c6f53e4239b8c2b209914065eb/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:10.649651 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e5ea37-d44b-473c-870b-236a9e968300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20bf4fe4ecc74735a2abdb5068b0785a, user_id=305767c6f53e4239b8c2b209914065eb, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:10.654257 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e5ea37-d44b-473c-870b-236a9e968300 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:10.660461 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf446420-9c99-4893-9665-ed9cba2d640c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:10.662348 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-bf446420-9c99-4893-9665-ed9cba2d640c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:10.676175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 955/1892] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:10 2022] PUT /identity/v3/projects/20bf4fe4ecc74735a2abdb5068b0785a/users/305767c6f53e4239b8c2b209914065eb/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:10.681270 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf446420-9c99-4893-9665-ed9cba2d640c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:10.682061 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b211173-6c09-4443-a2e6-713d540835d0 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:10.682418 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b211173-6c09-4443-a2e6-713d540835d0 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:10.682896 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b211173-6c09-4443-a2e6-713d540835d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:10.714239 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-9b211173-6c09-4443-a2e6-713d540835d0 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:10.716282 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-9b211173-6c09-4443-a2e6-713d540835d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:10.783250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e88adb-725e-46fa-88bf-889e9e029b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:10.784949 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e88adb-725e-46fa-88bf-889e9e029b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:10.785585 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e88adb-725e-46fa-88bf-889e9e029b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:10.786161 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e88adb-725e-46fa-88bf-889e9e029b9b admin admin] PATH_INFO: `/v3/users/7d34f6535e72410eb1cd234e58cb4141` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:10.788621 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e88adb-725e-46fa-88bf-889e9e029b9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d34f6535e72410eb1cd234e58cb4141)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:10.792503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e88adb-725e-46fa-88bf-889e9e029b9b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:10.802973 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 956/1893] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:10.817429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b211173-6c09-4443-a2e6-713d540835d0 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:10.820952 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-9b211173-6c09-4443-a2e6-713d540835d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:10.892630 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-68e88adb-725e-46fa-88bf-889e9e029b9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d34f6535e72410eb1cd234e58cb4141'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:10.908188 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-68e88adb-725e-46fa-88bf-889e9e029b9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d34f6535e72410eb1cd234e58cb4141'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:10.909339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-68e88adb-725e-46fa-88bf-889e9e029b9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d34f6535e72410eb1cd234e58cb4141'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:10.922878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-68e88adb-725e-46fa-88bf-889e9e029b9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d34f6535e72410eb1cd234e58cb4141'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:10.940769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-68e88adb-725e-46fa-88bf-889e9e029b9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d34f6535e72410eb1cd234e58cb4141'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:10.955864 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 938/1894] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:10 2022] DELETE /identity/v3/users/7d34f6535e72410eb1cd234e58cb4141 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:35:10.960105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd2468-bed5-46ed-8e97-4778f0b6a5db None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:10.960478 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd2468-bed5-46ed-8e97-4778f0b6a5db None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:10.961182 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd2468-bed5-46ed-8e97-4778f0b6a5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:10.963402 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b211173-6c09-4443-a2e6-713d540835d0 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:10.990391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-69fd2468-bed5-46ed-8e97-4778f0b6a5db None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:10.991607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-69fd2468-bed5-46ed-8e97-4778f0b6a5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:11.021522 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:11.042495 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1ffec-4906-4f43-a0b7-b4b0e2df494c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:11.043383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1ffec-4906-4f43-a0b7-b4b0e2df494c service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:11.043736 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1ffec-4906-4f43-a0b7-b4b0e2df494c service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:11.044330 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1ffec-4906-4f43-a0b7-b4b0e2df494c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:11.054471 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-d4f1ffec-4906-4f43-a0b7-b4b0e2df494c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:11.064607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 939/1895] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:35:11.074768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd2468-bed5-46ed-8e97-4778f0b6a5db None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:11.076373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-69fd2468-bed5-46ed-8e97-4778f0b6a5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:11.089521 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd2468-bed5-46ed-8e97-4778f0b6a5db None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:11.093557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5006a508-061c-402f-a8a9-837f1707a40e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:11.094268 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006a508-061c-402f-a8a9-837f1707a40e service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:11.094566 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006a508-061c-402f-a8a9-837f1707a40e service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:11.095954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006a508-061c-402f-a8a9-837f1707a40e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:11.103765 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:11.106907 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5006a508-061c-402f-a8a9-837f1707a40e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:11.110103 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5006a508-061c-402f-a8a9-837f1707a40e service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:11.126266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 940/1896] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:35:11.131266 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1ffec-4906-4f43-a0b7-b4b0e2df494c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:11.132922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1ffec-4906-4f43-a0b7-b4b0e2df494c service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:11.144375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 957/1897] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:11.577242 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5006a508-061c-402f-a8a9-837f1707a40e service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:11.579798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5006a508-061c-402f-a8a9-837f1707a40e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:11.593723 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5006a508-061c-402f-a8a9-837f1707a40e service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:11.599937 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8de395-85a5-4d72-b3a0-e91473e3411c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:11.600835 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8de395-85a5-4d72-b3a0-e91473e3411c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:11.601121 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8de395-85a5-4d72-b3a0-e91473e3411c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:11.601443 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8de395-85a5-4d72-b3a0-e91473e3411c admin admin] PATH_INFO: `/v3/projects/b43c41fb210f4ca99ac01be06202f5ee` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:11.602657 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8de395-85a5-4d72-b3a0-e91473e3411c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b43c41fb210f4ca99ac01be06202f5ee)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:11.604573 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8de395-85a5-4d72-b3a0-e91473e3411c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:11.668791 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1a8de395-85a5-4d72-b3a0-e91473e3411c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b43c41fb210f4ca99ac01be06202f5ee', 'request_id': 'req-1a8de395-85a5-4d72-b3a0-e91473e3411c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:11.679114 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1a8de395-85a5-4d72-b3a0-e91473e3411c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b43c41fb210f4ca99ac01be06202f5ee', 'request_id': 'req-1a8de395-85a5-4d72-b3a0-e91473e3411c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:11.686259 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1a8de395-85a5-4d72-b3a0-e91473e3411c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b43c41fb210f4ca99ac01be06202f5ee', 'request_id': 'req-1a8de395-85a5-4d72-b3a0-e91473e3411c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:11.697707 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1a8de395-85a5-4d72-b3a0-e91473e3411c admin admin] The token cache is being invalidate because project b43c41fb210f4ca99ac01be06202f5ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:11.697991 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1a8de395-85a5-4d72-b3a0-e91473e3411c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:11.698650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1a8de395-85a5-4d72-b3a0-e91473e3411c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:11.700386 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 941/1898] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:11 2022] DELETE /identity/v3/projects/b43c41fb210f4ca99ac01be06202f5ee => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:11.705047 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1ffec-4906-4f43-a0b7-b4b0e2df494c service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:11.715367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1ffec-4906-4f43-a0b7-b4b0e2df494c service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:11.793409 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f50d9a-4575-4b0c-b5d9-4e2b3bcdce13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:11.794450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f50d9a-4575-4b0c-b5d9-4e2b3bcdce13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:11.794934 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f50d9a-4575-4b0c-b5d9-4e2b3bcdce13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:11.795324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f50d9a-4575-4b0c-b5d9-4e2b3bcdce13 admin admin] PATH_INFO: `/v3/users/2782f8e82853499a882696d7df5c4e09` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:11.796926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f50d9a-4575-4b0c-b5d9-4e2b3bcdce13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2782f8e82853499a882696d7df5c4e09)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:11.799865 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f50d9a-4575-4b0c-b5d9-4e2b3bcdce13 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:11.863261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b6f50d9a-4575-4b0c-b5d9-4e2b3bcdce13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2782f8e82853499a882696d7df5c4e09'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:11.872196 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b6f50d9a-4575-4b0c-b5d9-4e2b3bcdce13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2782f8e82853499a882696d7df5c4e09'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:11.873044 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b6f50d9a-4575-4b0c-b5d9-4e2b3bcdce13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2782f8e82853499a882696d7df5c4e09'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:11.883180 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b6f50d9a-4575-4b0c-b5d9-4e2b3bcdce13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2782f8e82853499a882696d7df5c4e09'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:11.900271 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b6f50d9a-4575-4b0c-b5d9-4e2b3bcdce13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2782f8e82853499a882696d7df5c4e09'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:11.907414 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 958/1899] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:11 2022] DELETE /identity/v3/users/2782f8e82853499a882696d7df5c4e09 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:12.263875 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d8d62-82ac-4799-a22f-ad3c9cec0aa8 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:12.264281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d8d62-82ac-4799-a22f-ad3c9cec0aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:12.264648 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d8d62-82ac-4799-a22f-ad3c9cec0aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:12.297190 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-df8d8d62-82ac-4799-a22f-ad3c9cec0aa8 None None] MFA Rules not processed for user `305767c6f53e4239b8c2b209914065eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:12.299088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-df8d8d62-82ac-4799-a22f-ad3c9cec0aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:12.349270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f50d9a-4575-4b0c-b5d9-4e2b3bcdce13 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:12.351032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b6f50d9a-4575-4b0c-b5d9-4e2b3bcdce13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:12.368498 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:12.456123 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f50d9a-4575-4b0c-b5d9-4e2b3bcdce13 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:12.462671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c585e86-13f3-4cb2-9fcc-f7af641064db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:12.463826 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c585e86-13f3-4cb2-9fcc-f7af641064db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:12.464128 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c585e86-13f3-4cb2-9fcc-f7af641064db admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:12.464489 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c585e86-13f3-4cb2-9fcc-f7af641064db admin admin] PATH_INFO: `/v3/projects/c7ac7605b76045e6be617934d0246856` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:12.466569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c585e86-13f3-4cb2-9fcc-f7af641064db admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7ac7605b76045e6be617934d0246856)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:12.468111 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c585e86-13f3-4cb2-9fcc-f7af641064db admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:12.523577 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 942/1900] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:12.528943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-0c585e86-13f3-4cb2-9fcc-f7af641064db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7ac7605b76045e6be617934d0246856', 'request_id': 'req-0c585e86-13f3-4cb2-9fcc-f7af641064db'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:12.535142 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d8d62-82ac-4799-a22f-ad3c9cec0aa8 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:12.536511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-df8d8d62-82ac-4799-a22f-ad3c9cec0aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:12.536811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-0c585e86-13f3-4cb2-9fcc-f7af641064db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7ac7605b76045e6be617934d0246856', 'request_id': 'req-0c585e86-13f3-4cb2-9fcc-f7af641064db'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:12.543615 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-0c585e86-13f3-4cb2-9fcc-f7af641064db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7ac7605b76045e6be617934d0246856', 'request_id': 'req-0c585e86-13f3-4cb2-9fcc-f7af641064db'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:12.558026 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-0c585e86-13f3-4cb2-9fcc-f7af641064db admin admin] The token cache is being invalidate because project c7ac7605b76045e6be617934d0246856 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:12.558463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-0c585e86-13f3-4cb2-9fcc-f7af641064db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:12.559175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-0c585e86-13f3-4cb2-9fcc-f7af641064db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:12.561209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 959/1901] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:12 2022] DELETE /identity/v3/projects/c7ac7605b76045e6be617934d0246856 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:35:12.565184 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c585e86-13f3-4cb2-9fcc-f7af641064db admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:12.566743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0c585e86-13f3-4cb2-9fcc-f7af641064db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:12.608148 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d8d62-82ac-4799-a22f-ad3c9cec0aa8 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:12.633856 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:12.641222 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c585e86-13f3-4cb2-9fcc-f7af641064db admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:12.648793 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754afd5-9d78-4d74-ba8b-0639a7ff177f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:12.649283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754afd5-9d78-4d74-ba8b-0639a7ff177f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:12.649577 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754afd5-9d78-4d74-ba8b-0639a7ff177f admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:12.649862 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754afd5-9d78-4d74-ba8b-0639a7ff177f admin admin] PATH_INFO: `/v3/users/203bc15af47744c38d9e52f83203d801` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:12.651107 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754afd5-9d78-4d74-ba8b-0639a7ff177f admin admin] RBAC: Authorizing `identity:delete_user(user_id=203bc15af47744c38d9e52f83203d801)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:12.654540 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754afd5-9d78-4d74-ba8b-0639a7ff177f admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:12.672116 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa6038-544a-4e5d-8806-aab51f2a3a6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:12.673106 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa6038-544a-4e5d-8806-aab51f2a3a6d service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:12.673588 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa6038-544a-4e5d-8806-aab51f2a3a6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:12.674050 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa6038-544a-4e5d-8806-aab51f2a3a6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:12.677081 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c8fa6038-544a-4e5d-8806-aab51f2a3a6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:12.705980 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:12.715157 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a754afd5-9d78-4d74-ba8b-0639a7ff177f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '203bc15af47744c38d9e52f83203d801'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:12.728150 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a754afd5-9d78-4d74-ba8b-0639a7ff177f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '203bc15af47744c38d9e52f83203d801'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:12.729027 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a754afd5-9d78-4d74-ba8b-0639a7ff177f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '203bc15af47744c38d9e52f83203d801'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:12.746779 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a754afd5-9d78-4d74-ba8b-0639a7ff177f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '203bc15af47744c38d9e52f83203d801'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:12.753700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa6038-544a-4e5d-8806-aab51f2a3a6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:12.755664 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa6038-544a-4e5d-8806-aab51f2a3a6d service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:12.762234 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a754afd5-9d78-4d74-ba8b-0639a7ff177f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '203bc15af47744c38d9e52f83203d801'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:12.765689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 943/1902] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:12 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:35:12.769066 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 960/1903] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:12 2022] DELETE /identity/v3/users/203bc15af47744c38d9e52f83203d801 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:35:13.085292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa6038-544a-4e5d-8806-aab51f2a3a6d service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:13.130574 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa6038-544a-4e5d-8806-aab51f2a3a6d service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:13.279749 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43271f4b-8d5c-4191-bc3b-a97eb809adce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:13.280627 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43271f4b-8d5c-4191-bc3b-a97eb809adce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:13.280923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43271f4b-8d5c-4191-bc3b-a97eb809adce admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:13.293416 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43271f4b-8d5c-4191-bc3b-a97eb809adce admin admin] PATH_INFO: `/v3/projects/49b3c62472d449a0a0bc27c532f96912` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:13.297297 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43271f4b-8d5c-4191-bc3b-a97eb809adce admin admin] RBAC: Authorizing `identity:delete_project(project_id=49b3c62472d449a0a0bc27c532f96912)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:13.301334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43271f4b-8d5c-4191-bc3b-a97eb809adce admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:13.414362 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-43271f4b-8d5c-4191-bc3b-a97eb809adce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49b3c62472d449a0a0bc27c532f96912', 'request_id': 'req-43271f4b-8d5c-4191-bc3b-a97eb809adce'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:13.432179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-43271f4b-8d5c-4191-bc3b-a97eb809adce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49b3c62472d449a0a0bc27c532f96912', 'request_id': 'req-43271f4b-8d5c-4191-bc3b-a97eb809adce'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:13.458305 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-43271f4b-8d5c-4191-bc3b-a97eb809adce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49b3c62472d449a0a0bc27c532f96912', 'request_id': 'req-43271f4b-8d5c-4191-bc3b-a97eb809adce'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:13.479401 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-43271f4b-8d5c-4191-bc3b-a97eb809adce admin admin] The token cache is being invalidate because project 49b3c62472d449a0a0bc27c532f96912 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:13.479961 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-43271f4b-8d5c-4191-bc3b-a97eb809adce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:13.481353 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-43271f4b-8d5c-4191-bc3b-a97eb809adce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:13.487674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 944/1904] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:13 2022] DELETE /identity/v3/projects/49b3c62472d449a0a0bc27c532f96912 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:13.489189 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11f163-2f4a-4bb5-b732-8e492408e67f None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:13.489492 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11f163-2f4a-4bb5-b732-8e492408e67f None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:13.489782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11f163-2f4a-4bb5-b732-8e492408e67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:13.522154 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-4a11f163-2f4a-4bb5-b732-8e492408e67f None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:13.523196 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4a11f163-2f4a-4bb5-b732-8e492408e67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:13.649491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 961/1905] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:13.651231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35add5d1-f89f-419e-a29e-3a5f79deb664 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:13.651672 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35add5d1-f89f-419e-a29e-3a5f79deb664 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:13.651813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35add5d1-f89f-419e-a29e-3a5f79deb664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:13.675806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-35add5d1-f89f-419e-a29e-3a5f79deb664 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:13.677552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-35add5d1-f89f-419e-a29e-3a5f79deb664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:13.693299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 945/1906] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:13.695129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11f163-2f4a-4bb5-b732-8e492408e67f None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:13.707297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11f163-2f4a-4bb5-b732-8e492408e67f None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:13.712718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123d0f2-c90a-439b-93ab-d2be2ba7d716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:13.713559 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123d0f2-c90a-439b-93ab-d2be2ba7d716 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:13.713940 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123d0f2-c90a-439b-93ab-d2be2ba7d716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:13.714291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123d0f2-c90a-439b-93ab-d2be2ba7d716 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:13.715271 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123d0f2-c90a-439b-93ab-d2be2ba7d716 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:35:13.715673 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123d0f2-c90a-439b-93ab-d2be2ba7d716 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:13.718747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123d0f2-c90a-439b-93ab-d2be2ba7d716 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:13.739434 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 962/1907] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:35:13.740643 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35add5d1-f89f-419e-a29e-3a5f79deb664 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:13.746790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:13.755511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35add5d1-f89f-419e-a29e-3a5f79deb664 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:13.764568 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3f06a-403f-414d-9f6c-ae8e41494d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:13.775239 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3f06a-403f-414d-9f6c-ae8e41494d41 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:13.775893 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3f06a-403f-414d-9f6c-ae8e41494d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:13.776279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3f06a-403f-414d-9f6c-ae8e41494d41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:13.778032 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd3f06a-403f-414d-9f6c-ae8e41494d41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:13.785956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd3f06a-403f-414d-9f6c-ae8e41494d41 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:13.832586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 946/1908] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:13 2022] POST /identity/v3/projects => generated 387 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:13.834023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123d0f2-c90a-439b-93ab-d2be2ba7d716 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:13.846936 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123d0f2-c90a-439b-93ab-d2be2ba7d716 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:13.916334 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfdf0b-51de-4298-9efb-01343bab914d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:13.917242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfdf0b-51de-4298-9efb-01343bab914d admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:13.917722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfdf0b-51de-4298-9efb-01343bab914d admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:13.918092 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfdf0b-51de-4298-9efb-01343bab914d admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:13.918937 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acfdf0b-51de-4298-9efb-01343bab914d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:13.920883 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acfdf0b-51de-4298-9efb-01343bab914d admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:13.947976 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 963/1909] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:13 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:13.952238 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3f06a-403f-414d-9f6c-ae8e41494d41 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:13.964196 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3f06a-403f-414d-9f6c-ae8e41494d41 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:13.968933 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58c1e6-73af-4000-8329-27a37c310ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:13.969685 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58c1e6-73af-4000-8329-27a37c310ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:13.969973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58c1e6-73af-4000-8329-27a37c310ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:13.970438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58c1e6-73af-4000-8329-27a37c310ba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:13.971184 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb58c1e6-73af-4000-8329-27a37c310ba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:13.972783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb58c1e6-73af-4000-8329-27a37c310ba0 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:13.984368 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 947/1910] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:13.988533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfdf0b-51de-4298-9efb-01343bab914d admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:13.998704 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfdf0b-51de-4298-9efb-01343bab914d admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:14.003926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086e412-81bc-4edc-ae89-3855533baa58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:14.004582 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086e412-81bc-4edc-ae89-3855533baa58 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:14.004877 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086e412-81bc-4edc-ae89-3855533baa58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:14.005174 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086e412-81bc-4edc-ae89-3855533baa58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:14.005921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086e412-81bc-4edc-ae89-3855533baa58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:14.007485 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086e412-81bc-4edc-ae89-3855533baa58 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:14.018398 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 964/1911] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:14.022557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58c1e6-73af-4000-8329-27a37c310ba0 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:14.032561 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58c1e6-73af-4000-8329-27a37c310ba0 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:14.037451 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f3426-b47f-413a-b8d3-97c854b63e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:14.038362 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f3426-b47f-413a-b8d3-97c854b63e87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:14.038646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f3426-b47f-413a-b8d3-97c854b63e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:14.038924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f3426-b47f-413a-b8d3-97c854b63e87 admin admin] PATH_INFO: `/v3/projects/4a8672dbe753437fbd749a339dd3010f/users/3b73adaf89f54a8e93e3288503840732/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:14.053313 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f3426-b47f-413a-b8d3-97c854b63e87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a8672dbe753437fbd749a339dd3010f, user_id=3b73adaf89f54a8e93e3288503840732, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:14.055940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f3426-b47f-413a-b8d3-97c854b63e87 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:14.070105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 948/1912] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:14 2022] PUT /identity/v3/projects/4a8672dbe753437fbd749a339dd3010f/users/3b73adaf89f54a8e93e3288503840732/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:14.081938 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e0fdd-937a-4f21-b150-8c5462b07fa8 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:14.082312 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e0fdd-937a-4f21-b150-8c5462b07fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:14.082664 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e0fdd-937a-4f21-b150-8c5462b07fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:14.109380 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-556e0fdd-937a-4f21-b150-8c5462b07fa8 None None] MFA Rules not processed for user `3b73adaf89f54a8e93e3288503840732`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:14.111314 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-556e0fdd-937a-4f21-b150-8c5462b07fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:14.120333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:14.178693 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c11407-31d1-414c-a755-c7140c2e6efd None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:14.178990 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c11407-31d1-414c-a755-c7140c2e6efd None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:14.179276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c11407-31d1-414c-a755-c7140c2e6efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:14.180654 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 965/1913] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:14.184924 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e0fdd-937a-4f21-b150-8c5462b07fa8 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:14.197577 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-28c11407-31d1-414c-a755-c7140c2e6efd None None] MFA Rules not processed for user `305767c6f53e4239b8c2b209914065eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:14.198926 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-28c11407-31d1-414c-a755-c7140c2e6efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:14.259666 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e0fdd-937a-4f21-b150-8c5462b07fa8 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:14.270013 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6124003-53f4-416d-899a-bcd74de68692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:14.270690 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6124003-53f4-416d-899a-bcd74de68692 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:14.271207 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6124003-53f4-416d-899a-bcd74de68692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:14.271509 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6124003-53f4-416d-899a-bcd74de68692 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:14.272350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6124003-53f4-416d-899a-bcd74de68692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:14.274327 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6124003-53f4-416d-899a-bcd74de68692 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:14.278771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 949/1914] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:14 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:35:14.310760 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 966/1915] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:14 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:35:14.314930 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c11407-31d1-414c-a755-c7140c2e6efd None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:14.327908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c11407-31d1-414c-a755-c7140c2e6efd None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:14.412887 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8672a80-c79a-4bdf-ba1c-c06f32b604d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:14.415758 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8672a80-c79a-4bdf-ba1c-c06f32b604d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:14.416196 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8672a80-c79a-4bdf-ba1c-c06f32b604d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:14.417032 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8672a80-c79a-4bdf-ba1c-c06f32b604d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:14.418440 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8672a80-c79a-4bdf-ba1c-c06f32b604d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:14.422203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8672a80-c79a-4bdf-ba1c-c06f32b604d3 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:14.443582 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 950/1916] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:14 2022] POST /identity/v3/users => generated 299 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:35:14.447620 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6124003-53f4-416d-899a-bcd74de68692 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:14.459983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6124003-53f4-416d-899a-bcd74de68692 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:14.464781 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb3c8e-fd60-4394-b653-0fe2f84508ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:14.465722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb3c8e-fd60-4394-b653-0fe2f84508ef admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:14.466011 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb3c8e-fd60-4394-b653-0fe2f84508ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:14.466294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb3c8e-fd60-4394-b653-0fe2f84508ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:14.467027 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb3c8e-fd60-4394-b653-0fe2f84508ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:14.468624 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb3c8e-fd60-4394-b653-0fe2f84508ef admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:14.481348 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 967/1917] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:14.486313 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8672a80-c79a-4bdf-ba1c-c06f32b604d3 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:14.497107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8672a80-c79a-4bdf-ba1c-c06f32b604d3 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:14.502615 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501677d-5446-4eeb-a5d2-9b1f492035cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:14.503549 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501677d-5446-4eeb-a5d2-9b1f492035cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:14.503840 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501677d-5446-4eeb-a5d2-9b1f492035cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:14.504146 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501677d-5446-4eeb-a5d2-9b1f492035cf admin admin] PATH_INFO: `/v3/projects/a6829040f76b4cb495b1d0f6f50811ab/users/e4e4f3c3557741a5bac7316206cc851b/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:14.520380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7501677d-5446-4eeb-a5d2-9b1f492035cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6829040f76b4cb495b1d0f6f50811ab, user_id=e4e4f3c3557741a5bac7316206cc851b, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:14.523365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7501677d-5446-4eeb-a5d2-9b1f492035cf admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:14.537388 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 951/1918] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:14 2022] PUT /identity/v3/projects/a6829040f76b4cb495b1d0f6f50811ab/users/e4e4f3c3557741a5bac7316206cc851b/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:14.541437 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb3c8e-fd60-4394-b653-0fe2f84508ef admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:14.552559 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb3c8e-fd60-4394-b653-0fe2f84508ef admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:14.612862 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f722439d-ae2f-4098-aa84-d3795f2ac4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:14.613583 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722439d-ae2f-4098-aa84-d3795f2ac4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:14.613878 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722439d-ae2f-4098-aa84-d3795f2ac4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:14.614164 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722439d-ae2f-4098-aa84-d3795f2ac4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:14.614906 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f722439d-ae2f-4098-aa84-d3795f2ac4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:14.616545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f722439d-ae2f-4098-aa84-d3795f2ac4d9 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:14.627650 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 968/1919] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:14.631909 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501677d-5446-4eeb-a5d2-9b1f492035cf admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:14.642570 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501677d-5446-4eeb-a5d2-9b1f492035cf admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:14.647801 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74cd387-3195-4757-9a54-33ce8b29788d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:14.648721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cd387-3195-4757-9a54-33ce8b29788d admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:14.649016 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cd387-3195-4757-9a54-33ce8b29788d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:14.649328 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cd387-3195-4757-9a54-33ce8b29788d admin admin] PATH_INFO: `/v3/projects/a6829040f76b4cb495b1d0f6f50811ab/users/e4e4f3c3557741a5bac7316206cc851b/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:14.651452 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74cd387-3195-4757-9a54-33ce8b29788d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6829040f76b4cb495b1d0f6f50811ab, user_id=e4e4f3c3557741a5bac7316206cc851b, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:14.654172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74cd387-3195-4757-9a54-33ce8b29788d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:14.667038 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 952/1920] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:14 2022] PUT /identity/v3/projects/a6829040f76b4cb495b1d0f6f50811ab/users/e4e4f3c3557741a5bac7316206cc851b/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:14.681948 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf6213-d927-45ca-ba3f-7a110714787a None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:14.682430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf6213-d927-45ca-ba3f-7a110714787a None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:14.683034 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf6213-d927-45ca-ba3f-7a110714787a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:14.720827 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-45cf6213-d927-45ca-ba3f-7a110714787a None None] MFA Rules not processed for user `e4e4f3c3557741a5bac7316206cc851b`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:14.722124 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-45cf6213-d927-45ca-ba3f-7a110714787a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:14.732952 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:14.804210 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 969/1921] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:14.832345 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74cd387-3195-4757-9a54-33ce8b29788d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:14.834558 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d74cd387-3195-4757-9a54-33ce8b29788d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:14.840211 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:14.923537 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74cd387-3195-4757-9a54-33ce8b29788d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:14.932691 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498b911-e8d7-4b30-ae2e-10489f8fe655 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:14.934175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498b911-e8d7-4b30-ae2e-10489f8fe655 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:14.934665 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498b911-e8d7-4b30-ae2e-10489f8fe655 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:14.935271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498b911-e8d7-4b30-ae2e-10489f8fe655 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:14.950497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498b911-e8d7-4b30-ae2e-10489f8fe655 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:14.952056 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498b911-e8d7-4b30-ae2e-10489f8fe655 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:14.970259 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 953/1922] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:15.183239 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf6213-d927-45ca-ba3f-7a110714787a None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:15.281040 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf6213-d927-45ca-ba3f-7a110714787a None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:15.287031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6dde9-336d-413b-9f08-6af27397e761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:15.300367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6dde9-336d-413b-9f08-6af27397e761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:15.303081 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6dde9-336d-413b-9f08-6af27397e761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:15.303247 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6dde9-336d-413b-9f08-6af27397e761 admin admin] PATH_INFO: `/v3/users/3b73adaf89f54a8e93e3288503840732` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:15.305404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6dde9-336d-413b-9f08-6af27397e761 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b73adaf89f54a8e93e3288503840732)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:15.311238 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6dde9-336d-413b-9f08-6af27397e761 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:15.433889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-99a6dde9-336d-413b-9f08-6af27397e761 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b73adaf89f54a8e93e3288503840732'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:15.445840 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-99a6dde9-336d-413b-9f08-6af27397e761 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b73adaf89f54a8e93e3288503840732'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:15.446500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-99a6dde9-336d-413b-9f08-6af27397e761 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b73adaf89f54a8e93e3288503840732'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:15.456069 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-99a6dde9-336d-413b-9f08-6af27397e761 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b73adaf89f54a8e93e3288503840732'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:15.473602 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-99a6dde9-336d-413b-9f08-6af27397e761 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b73adaf89f54a8e93e3288503840732'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:15.481449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 970/1923] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:15 2022] DELETE /identity/v3/users/3b73adaf89f54a8e93e3288503840732 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:15.486046 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22b0a1-f8b3-4bcb-9c09-da26e83b5c27 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:15.486339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22b0a1-f8b3-4bcb-9c09-da26e83b5c27 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:15.486629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22b0a1-f8b3-4bcb-9c09-da26e83b5c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:15.518449 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-fc22b0a1-f8b3-4bcb-9c09-da26e83b5c27 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:15.519706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fc22b0a1-f8b3-4bcb-9c09-da26e83b5c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:15.595986 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 954/1924] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:15.602639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6dde9-336d-413b-9f08-6af27397e761 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:15.604190 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-99a6dde9-336d-413b-9f08-6af27397e761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:15.636344 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:15.667116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6dde9-336d-413b-9f08-6af27397e761 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:15.670790 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f76f27-a4f1-4f71-a05e-d0a7fa59f0ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:15.671455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f76f27-a4f1-4f71-a05e-d0a7fa59f0ce service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:15.671776 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f76f27-a4f1-4f71-a05e-d0a7fa59f0ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:15.672216 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f76f27-a4f1-4f71-a05e-d0a7fa59f0ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:15.683970 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f76f27-a4f1-4f71-a05e-d0a7fa59f0ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:15.685379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f76f27-a4f1-4f71-a05e-d0a7fa59f0ce service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:15.695153 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 971/1925] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:15.988015 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22b0a1-f8b3-4bcb-9c09-da26e83b5c27 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:15.989508 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fc22b0a1-f8b3-4bcb-9c09-da26e83b5c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:16.001715 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22b0a1-f8b3-4bcb-9c09-da26e83b5c27 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:16.007138 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999861c-2c49-4974-a6ea-34038b2cf0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:16.008420 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999861c-2c49-4974-a6ea-34038b2cf0b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:16.008779 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999861c-2c49-4974-a6ea-34038b2cf0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:16.009129 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999861c-2c49-4974-a6ea-34038b2cf0b7 admin admin] PATH_INFO: `/v3/projects/4a8672dbe753437fbd749a339dd3010f` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:16.012416 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f999861c-2c49-4974-a6ea-34038b2cf0b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a8672dbe753437fbd749a339dd3010f)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:16.015137 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f999861c-2c49-4974-a6ea-34038b2cf0b7 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:16.067763 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f999861c-2c49-4974-a6ea-34038b2cf0b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a8672dbe753437fbd749a339dd3010f', 'request_id': 'req-f999861c-2c49-4974-a6ea-34038b2cf0b7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:16.075555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f999861c-2c49-4974-a6ea-34038b2cf0b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a8672dbe753437fbd749a339dd3010f', 'request_id': 'req-f999861c-2c49-4974-a6ea-34038b2cf0b7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:16.082290 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f999861c-2c49-4974-a6ea-34038b2cf0b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a8672dbe753437fbd749a339dd3010f', 'request_id': 'req-f999861c-2c49-4974-a6ea-34038b2cf0b7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:16.093557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f999861c-2c49-4974-a6ea-34038b2cf0b7 admin admin] The token cache is being invalidate because project 4a8672dbe753437fbd749a339dd3010f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:16.093850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f999861c-2c49-4974-a6ea-34038b2cf0b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:16.095056 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f999861c-2c49-4974-a6ea-34038b2cf0b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:16.097003 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 955/1926] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:15 2022] DELETE /identity/v3/projects/4a8672dbe753437fbd749a339dd3010f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:16.101080 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f76f27-a4f1-4f71-a05e-d0a7fa59f0ce service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:16.112170 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f76f27-a4f1-4f71-a05e-d0a7fa59f0ce service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:16.196987 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b433a-faca-4b6d-aecb-5e23561e3275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:16.198182 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b433a-faca-4b6d-aecb-5e23561e3275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:16.200647 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b433a-faca-4b6d-aecb-5e23561e3275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:16.200647 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b433a-faca-4b6d-aecb-5e23561e3275 admin admin] PATH_INFO: `/v3/users/e4e4f3c3557741a5bac7316206cc851b` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:16.201607 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b433a-faca-4b6d-aecb-5e23561e3275 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4e4f3c3557741a5bac7316206cc851b)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:16.205908 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b433a-faca-4b6d-aecb-5e23561e3275 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:16.264240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-fa8b433a-faca-4b6d-aecb-5e23561e3275 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4e4f3c3557741a5bac7316206cc851b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:16.273516 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-fa8b433a-faca-4b6d-aecb-5e23561e3275 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4e4f3c3557741a5bac7316206cc851b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:16.274161 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-fa8b433a-faca-4b6d-aecb-5e23561e3275 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4e4f3c3557741a5bac7316206cc851b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:16.283491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-fa8b433a-faca-4b6d-aecb-5e23561e3275 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4e4f3c3557741a5bac7316206cc851b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:16.303995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-fa8b433a-faca-4b6d-aecb-5e23561e3275 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4e4f3c3557741a5bac7316206cc851b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:16.311564 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 972/1927] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:16 2022] DELETE /identity/v3/users/e4e4f3c3557741a5bac7316206cc851b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:16.734332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999861c-2c49-4974-a6ea-34038b2cf0b7 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:16.736238 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f999861c-2c49-4974-a6ea-34038b2cf0b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:16.799935 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999861c-2c49-4974-a6ea-34038b2cf0b7 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:16.840304 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:16.898110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d2faa-71a0-4e77-9497-505ec35bf7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:16.899253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d2faa-71a0-4e77-9497-505ec35bf7f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:16.899863 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d2faa-71a0-4e77-9497-505ec35bf7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:16.899984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d2faa-71a0-4e77-9497-505ec35bf7f2 admin admin] PATH_INFO: `/v3/projects/a6829040f76b4cb495b1d0f6f50811ab` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:16.901947 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d2faa-71a0-4e77-9497-505ec35bf7f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6829040f76b4cb495b1d0f6f50811ab)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:16.904285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d2faa-71a0-4e77-9497-505ec35bf7f2 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:16.976945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ff7d2faa-71a0-4e77-9497-505ec35bf7f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6829040f76b4cb495b1d0f6f50811ab', 'request_id': 'req-ff7d2faa-71a0-4e77-9497-505ec35bf7f2'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:16.990184 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ff7d2faa-71a0-4e77-9497-505ec35bf7f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6829040f76b4cb495b1d0f6f50811ab', 'request_id': 'req-ff7d2faa-71a0-4e77-9497-505ec35bf7f2'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:16.998928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ff7d2faa-71a0-4e77-9497-505ec35bf7f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6829040f76b4cb495b1d0f6f50811ab', 'request_id': 'req-ff7d2faa-71a0-4e77-9497-505ec35bf7f2'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:17.012043 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ff7d2faa-71a0-4e77-9497-505ec35bf7f2 admin admin] The token cache is being invalidate because project a6829040f76b4cb495b1d0f6f50811ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:17.012344 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ff7d2faa-71a0-4e77-9497-505ec35bf7f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:17.013029 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ff7d2faa-71a0-4e77-9497-505ec35bf7f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:17.018661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 956/1928] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:16 2022] DELETE /identity/v3/projects/a6829040f76b4cb495b1d0f6f50811ab => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:17.022119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94b548-6311-4a67-a417-5114f981ce29 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:17.022865 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94b548-6311-4a67-a417-5114f981ce29 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:17.023483 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94b548-6311-4a67-a417-5114f981ce29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:17.054734 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-6d94b548-6311-4a67-a417-5114f981ce29 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:17.056531 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-6d94b548-6311-4a67-a417-5114f981ce29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:17.135824 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:17.183878 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 973/1929] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:17.212511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a49018-9149-41a1-87a3-ad670d120cde None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:17.212826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a49018-9149-41a1-87a3-ad670d120cde None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:17.213232 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a49018-9149-41a1-87a3-ad670d120cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:17.255512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-12a49018-9149-41a1-87a3-ad670d120cde None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:17.256910 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-12a49018-9149-41a1-87a3-ad670d120cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:17.275963 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94b548-6311-4a67-a417-5114f981ce29 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:17.277770 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-6d94b548-6311-4a67-a417-5114f981ce29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:17.281724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 957/1930] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:17.290453 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a49018-9149-41a1-87a3-ad670d120cde None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:17.292638 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94b548-6311-4a67-a417-5114f981ce29 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:17.299902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d34d61-56a3-4eee-b4a7-f94a546ec885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:17.304411 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a49018-9149-41a1-87a3-ad670d120cde None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:17.311602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1578ce1-65a8-4074-b37c-3aa523b984a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:17.312432 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1578ce1-65a8-4074-b37c-3aa523b984a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:17.312646 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d34d61-56a3-4eee-b4a7-f94a546ec885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:17.312749 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1578ce1-65a8-4074-b37c-3aa523b984a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:17.312986 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1578ce1-65a8-4074-b37c-3aa523b984a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:17.313129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d34d61-56a3-4eee-b4a7-f94a546ec885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:17.313926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d34d61-56a3-4eee-b4a7-f94a546ec885 admin admin] PATH_INFO: `/v3/users/1a858dbccddf49a285dafcc972a025cc` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:17.314612 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1578ce1-65a8-4074-b37c-3aa523b984a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:35:17.314931 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1578ce1-65a8-4074-b37c-3aa523b984a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:17.315662 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d34d61-56a3-4eee-b4a7-f94a546ec885 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a858dbccddf49a285dafcc972a025cc)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:17.316889 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1578ce1-65a8-4074-b37c-3aa523b984a8 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:17.318800 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d34d61-56a3-4eee-b4a7-f94a546ec885 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:17.333940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 958/1931] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:35:17.338992 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1578ce1-65a8-4074-b37c-3aa523b984a8 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:17.353017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1578ce1-65a8-4074-b37c-3aa523b984a8 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:17.362410 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbb108-b57e-436e-90ad-80f1cdf53e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:17.363746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbb108-b57e-436e-90ad-80f1cdf53e92 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:17.364044 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbb108-b57e-436e-90ad-80f1cdf53e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:17.364324 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbb108-b57e-436e-90ad-80f1cdf53e92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:17.365035 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbb108-b57e-436e-90ad-80f1cdf53e92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:17.367058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbb108-b57e-436e-90ad-80f1cdf53e92 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:17.380821 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-e4d34d61-56a3-4eee-b4a7-f94a546ec885 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a858dbccddf49a285dafcc972a025cc'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:17.391747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-e4d34d61-56a3-4eee-b4a7-f94a546ec885 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a858dbccddf49a285dafcc972a025cc'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:17.392550 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-e4d34d61-56a3-4eee-b4a7-f94a546ec885 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a858dbccddf49a285dafcc972a025cc'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:17.397638 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 959/1932] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:17 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:35:17.401806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbb108-b57e-436e-90ad-80f1cdf53e92 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:17.406352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-e4d34d61-56a3-4eee-b4a7-f94a546ec885 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a858dbccddf49a285dafcc972a025cc'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:17.424103 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-e4d34d61-56a3-4eee-b4a7-f94a546ec885 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a858dbccddf49a285dafcc972a025cc'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:17.431069 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 974/1933] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:17 2022] DELETE /identity/v3/users/1a858dbccddf49a285dafcc972a025cc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:17.435116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac60ce0-c627-4d6e-bf63-95c5eca8c09c None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:17.435410 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac60ce0-c627-4d6e-bf63-95c5eca8c09c None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:17.435711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac60ce0-c627-4d6e-bf63-95c5eca8c09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:17.470832 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-5ac60ce0-c627-4d6e-bf63-95c5eca8c09c None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:17.474131 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-5ac60ce0-c627-4d6e-bf63-95c5eca8c09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:17.499729 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbb108-b57e-436e-90ad-80f1cdf53e92 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:17.540149 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:17.574258 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 975/1934] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:35:17.585350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac60ce0-c627-4d6e-bf63-95c5eca8c09c None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:17.587221 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-5ac60ce0-c627-4d6e-bf63-95c5eca8c09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:17.604492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bacb8-aeca-4eed-993d-e2d1ad3f92c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:17.605764 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bacb8-aeca-4eed-993d-e2d1ad3f92c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:17.606197 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bacb8-aeca-4eed-993d-e2d1ad3f92c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:17.606623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bacb8-aeca-4eed-993d-e2d1ad3f92c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:17.607728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73bacb8-aeca-4eed-993d-e2d1ad3f92c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:17.611215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73bacb8-aeca-4eed-993d-e2d1ad3f92c9 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:17.643384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 960/1935] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:17 2022] POST /identity/v3/users => generated 289 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:35:17.647905 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bacb8-aeca-4eed-993d-e2d1ad3f92c9 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:17.650014 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b73bacb8-aeca-4eed-993d-e2d1ad3f92c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:17.665479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bacb8-aeca-4eed-993d-e2d1ad3f92c9 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:17.673491 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f24570-2c45-4c1d-b3d7-b24c4fa22441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:17.674548 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f24570-2c45-4c1d-b3d7-b24c4fa22441 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:17.674878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f24570-2c45-4c1d-b3d7-b24c4fa22441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:17.675495 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f24570-2c45-4c1d-b3d7-b24c4fa22441 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:17.676144 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f24570-2c45-4c1d-b3d7-b24c4fa22441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:17.677934 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f24570-2c45-4c1d-b3d7-b24c4fa22441 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:17.690636 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 961/1936] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:17.695470 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f24570-2c45-4c1d-b3d7-b24c4fa22441 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:17.700996 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac60ce0-c627-4d6e-bf63-95c5eca8c09c None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:17.706343 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec20047-1d40-4484-bba0-767ad54611ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:17.707434 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec20047-1d40-4484-bba0-767ad54611ea service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:17.707950 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec20047-1d40-4484-bba0-767ad54611ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:17.708133 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f24570-2c45-4c1d-b3d7-b24c4fa22441 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:17.708754 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec20047-1d40-4484-bba0-767ad54611ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:17.714508 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ffbe6-a6c0-4aa1-bac0-5e037739d698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:17.715590 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ffbe6-a6c0-4aa1-bac0-5e037739d698 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:17.715950 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ffbe6-a6c0-4aa1-bac0-5e037739d698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:17.716389 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ffbe6-a6c0-4aa1-bac0-5e037739d698 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:17.717712 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ffbe6-a6c0-4aa1-bac0-5e037739d698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:17.720035 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ffbe6-a6c0-4aa1-bac0-5e037739d698 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:17.736540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 962/1937] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:17.738151 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ffbe6-a6c0-4aa1-bac0-5e037739d698 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:17.744029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec20047-1d40-4484-bba0-767ad54611ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:17.745685 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec20047-1d40-4484-bba0-767ad54611ea service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:17.748937 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ffbe6-a6c0-4aa1-bac0-5e037739d698 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:17.754190 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cfb1a5-e4f5-43ad-bf19-a67dcc09bf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:17.755021 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cfb1a5-e4f5-43ad-bf19-a67dcc09bf0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:17.755324 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cfb1a5-e4f5-43ad-bf19-a67dcc09bf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:17.755842 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cfb1a5-e4f5-43ad-bf19-a67dcc09bf0c admin admin] PATH_INFO: `/v3/projects/81bac1058fd248d897738ecba112fdf0/users/79fe74e3cab54be88f66756cea434356/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:17.771325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 976/1938] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:17.775401 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cfb1a5-e4f5-43ad-bf19-a67dcc09bf0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=81bac1058fd248d897738ecba112fdf0, user_id=79fe74e3cab54be88f66756cea434356, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:17.779226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cfb1a5-e4f5-43ad-bf19-a67dcc09bf0c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:17.819897 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 963/1939] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:17 2022] PUT /identity/v3/projects/81bac1058fd248d897738ecba112fdf0/users/79fe74e3cab54be88f66756cea434356/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:17.821625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da02fe-3027-4c48-ae7a-a1a3c3459772 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:17.821920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da02fe-3027-4c48-ae7a-a1a3c3459772 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:17.822233 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da02fe-3027-4c48-ae7a-a1a3c3459772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:17.830527 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:17.876047 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-58da02fe-3027-4c48-ae7a-a1a3c3459772 None None] MFA Rules not processed for user `79fe74e3cab54be88f66756cea434356`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:17.877817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-58da02fe-3027-4c48-ae7a-a1a3c3459772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:18.045208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 977/1940] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:18.048486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cfb1a5-e4f5-43ad-bf19-a67dcc09bf0c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:18.160354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cfb1a5-e4f5-43ad-bf19-a67dcc09bf0c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:18.166677 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1771e0b-cef2-44b0-84fd-57ae24078a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:18.167380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1771e0b-cef2-44b0-84fd-57ae24078a60 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:18.167675 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1771e0b-cef2-44b0-84fd-57ae24078a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:18.167960 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1771e0b-cef2-44b0-84fd-57ae24078a60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:18.168951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1771e0b-cef2-44b0-84fd-57ae24078a60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:18.171036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1771e0b-cef2-44b0-84fd-57ae24078a60 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:18.212476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 964/1941] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:18 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:18.216864 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da02fe-3027-4c48-ae7a-a1a3c3459772 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:18.244020 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da02fe-3027-4c48-ae7a-a1a3c3459772 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:18.372874 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814fa27-8964-4c97-bee8-188eadbd4d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:18.374074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814fa27-8964-4c97-bee8-188eadbd4d93 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:18.374620 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814fa27-8964-4c97-bee8-188eadbd4d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:18.375100 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814fa27-8964-4c97-bee8-188eadbd4d93 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:18.376125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814fa27-8964-4c97-bee8-188eadbd4d93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:18.381089 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814fa27-8964-4c97-bee8-188eadbd4d93 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:18.418989 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 978/1942] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:18 2022] POST /identity/v3/users => generated 290 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:18.424339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1771e0b-cef2-44b0-84fd-57ae24078a60 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:18.435921 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1771e0b-cef2-44b0-84fd-57ae24078a60 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:18.442342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063875a2-c394-466e-9b04-6c56f128522a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:18.443107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063875a2-c394-466e-9b04-6c56f128522a admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:18.443400 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063875a2-c394-466e-9b04-6c56f128522a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:18.443682 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063875a2-c394-466e-9b04-6c56f128522a admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:18.444496 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063875a2-c394-466e-9b04-6c56f128522a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:18.447679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063875a2-c394-466e-9b04-6c56f128522a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:18.484068 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 965/1943] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:18.486840 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814fa27-8964-4c97-bee8-188eadbd4d93 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:18.504107 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814fa27-8964-4c97-bee8-188eadbd4d93 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:18.521081 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aca802-45d9-4744-b3fc-05c0c0597cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:18.521986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aca802-45d9-4744-b3fc-05c0c0597cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:18.522380 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aca802-45d9-4744-b3fc-05c0c0597cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:18.522809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aca802-45d9-4744-b3fc-05c0c0597cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:18.523659 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aca802-45d9-4744-b3fc-05c0c0597cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:18.529221 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aca802-45d9-4744-b3fc-05c0c0597cd8 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:18.554045 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 979/1944] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:18.556195 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063875a2-c394-466e-9b04-6c56f128522a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:18.572861 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063875a2-c394-466e-9b04-6c56f128522a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:18.586912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d9d51-4a53-405c-9196-60dad2b4b165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:18.589569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d9d51-4a53-405c-9196-60dad2b4b165 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:18.589569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d9d51-4a53-405c-9196-60dad2b4b165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:18.589569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d9d51-4a53-405c-9196-60dad2b4b165 admin admin] PATH_INFO: `/v3/projects/88dbc9ed2dbb4ed1950a68407f014d35/users/97722ed40cc4451d81ef267b18dbb51a/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:18.613758 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d9d51-4a53-405c-9196-60dad2b4b165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88dbc9ed2dbb4ed1950a68407f014d35, user_id=97722ed40cc4451d81ef267b18dbb51a, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:18.617066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d9d51-4a53-405c-9196-60dad2b4b165 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:18.650740 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 966/1945] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:18 2022] PUT /identity/v3/projects/88dbc9ed2dbb4ed1950a68407f014d35/users/97722ed40cc4451d81ef267b18dbb51a/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:18.652812 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de21e5-6dba-4cd2-a1a4-3c750e1ae13c None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:18.653119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de21e5-6dba-4cd2-a1a4-3c750e1ae13c None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:18.653561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de21e5-6dba-4cd2-a1a4-3c750e1ae13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:18.710100 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-61de21e5-6dba-4cd2-a1a4-3c750e1ae13c None None] MFA Rules not processed for user `97722ed40cc4451d81ef267b18dbb51a`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:18.712123 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-61de21e5-6dba-4cd2-a1a4-3c750e1ae13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:18.821753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 980/1946] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:18.826268 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d9d51-4a53-405c-9196-60dad2b4b165 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:18.855175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:18.916986 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d9d51-4a53-405c-9196-60dad2b4b165 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:18.923774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc10d4-e70a-4259-a5d2-161146d96210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:18.924774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc10d4-e70a-4259-a5d2-161146d96210 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:18.925115 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc10d4-e70a-4259-a5d2-161146d96210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:18.925364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc10d4-e70a-4259-a5d2-161146d96210 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:18.926079 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cc10d4-e70a-4259-a5d2-161146d96210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:18.929694 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cc10d4-e70a-4259-a5d2-161146d96210 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:18.974378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 967/1947] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:18 2022] POST /identity/v3/projects => generated 379 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:18.988068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de21e5-6dba-4cd2-a1a4-3c750e1ae13c None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:18.994832 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:19.002595 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de21e5-6dba-4cd2-a1a4-3c750e1ae13c None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:19.142916 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12a796-d060-47f5-93e5-ab45065dac66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:19.143632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12a796-d060-47f5-93e5-ab45065dac66 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:19.143926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12a796-d060-47f5-93e5-ab45065dac66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:19.147061 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12a796-d060-47f5-93e5-ab45065dac66 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:19.148242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa12a796-d060-47f5-93e5-ab45065dac66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:19.152609 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa12a796-d060-47f5-93e5-ab45065dac66 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:19.196482 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 981/1948] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:18 2022] POST /identity/v3/users => generated 296 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:19.201293 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc10d4-e70a-4259-a5d2-161146d96210 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:19.216307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc10d4-e70a-4259-a5d2-161146d96210 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:19.221833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45dec1-869d-4a94-b7b6-dfcac5efbc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:19.222612 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45dec1-869d-4a94-b7b6-dfcac5efbc27 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:19.222969 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45dec1-869d-4a94-b7b6-dfcac5efbc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:19.223321 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45dec1-869d-4a94-b7b6-dfcac5efbc27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:19.224135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45dec1-869d-4a94-b7b6-dfcac5efbc27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:19.225864 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45dec1-869d-4a94-b7b6-dfcac5efbc27 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:19.260015 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 968/1949] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:19.265749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12a796-d060-47f5-93e5-ab45065dac66 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:19.281320 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12a796-d060-47f5-93e5-ab45065dac66 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:19.292077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60242e27-229e-4e18-b03e-f60ab4e96fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:19.293689 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60242e27-229e-4e18-b03e-f60ab4e96fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:19.293987 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60242e27-229e-4e18-b03e-f60ab4e96fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:19.294281 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60242e27-229e-4e18-b03e-f60ab4e96fbc admin admin] PATH_INFO: `/v3/projects/430804382c704b7d9688bcbc8bfd0c08/users/f9256ac4bd704680bab981f8c2d3aa0c/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:19.334319 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60242e27-229e-4e18-b03e-f60ab4e96fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=430804382c704b7d9688bcbc8bfd0c08, user_id=f9256ac4bd704680bab981f8c2d3aa0c, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:19.339566 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60242e27-229e-4e18-b03e-f60ab4e96fbc admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:19.340780 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45dec1-869d-4a94-b7b6-dfcac5efbc27 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:19.342568 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2f45dec1-869d-4a94-b7b6-dfcac5efbc27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:19.358383 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45dec1-869d-4a94-b7b6-dfcac5efbc27 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:19.370195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 982/1950] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:19 2022] PUT /identity/v3/projects/430804382c704b7d9688bcbc8bfd0c08/users/f9256ac4bd704680bab981f8c2d3aa0c/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:19.372613 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344249e3-9489-4aaa-86ca-60063c5a3955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:19.373841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344249e3-9489-4aaa-86ca-60063c5a3955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:19.373970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344249e3-9489-4aaa-86ca-60063c5a3955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:19.374227 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344249e3-9489-4aaa-86ca-60063c5a3955 admin admin] PATH_INFO: `/v3/projects/630ce45e2ddb42e582e52cd42f2920b2` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:19.376244 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344249e3-9489-4aaa-86ca-60063c5a3955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=630ce45e2ddb42e582e52cd42f2920b2)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:19.381253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60242e27-229e-4e18-b03e-f60ab4e96fbc admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:19.388104 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344249e3-9489-4aaa-86ca-60063c5a3955 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:19.495478 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-344249e3-9489-4aaa-86ca-60063c5a3955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '630ce45e2ddb42e582e52cd42f2920b2', 'request_id': 'req-344249e3-9489-4aaa-86ca-60063c5a3955'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:19.529693 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60242e27-229e-4e18-b03e-f60ab4e96fbc admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:19.549274 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-344249e3-9489-4aaa-86ca-60063c5a3955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '630ce45e2ddb42e582e52cd42f2920b2', 'request_id': 'req-344249e3-9489-4aaa-86ca-60063c5a3955'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:19.566074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-344249e3-9489-4aaa-86ca-60063c5a3955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '630ce45e2ddb42e582e52cd42f2920b2', 'request_id': 'req-344249e3-9489-4aaa-86ca-60063c5a3955'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:19.614469 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-344249e3-9489-4aaa-86ca-60063c5a3955 admin admin] The token cache is being invalidate because project 630ce45e2ddb42e582e52cd42f2920b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:19.614993 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-344249e3-9489-4aaa-86ca-60063c5a3955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:19.616090 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-344249e3-9489-4aaa-86ca-60063c5a3955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:19.618363 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 969/1951] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:19 2022] DELETE /identity/v3/projects/630ce45e2ddb42e582e52cd42f2920b2 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:19.623777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344249e3-9489-4aaa-86ca-60063c5a3955 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:19.626758 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-344249e3-9489-4aaa-86ca-60063c5a3955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:19.673427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727744ca-6b72-496c-a651-2f09649ac005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:19.674444 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727744ca-6b72-496c-a651-2f09649ac005 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:19.674792 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727744ca-6b72-496c-a651-2f09649ac005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:19.675109 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727744ca-6b72-496c-a651-2f09649ac005 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:19.675770 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727744ca-6b72-496c-a651-2f09649ac005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:19.677433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727744ca-6b72-496c-a651-2f09649ac005 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:19.706311 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 983/1952] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:19.710734 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727744ca-6b72-496c-a651-2f09649ac005 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:19.715474 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-727744ca-6b72-496c-a651-2f09649ac005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:19.749135 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727744ca-6b72-496c-a651-2f09649ac005 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:19.765912 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02629e4f-646e-4868-8698-bae6ed7c3e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:19.767623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02629e4f-646e-4868-8698-bae6ed7c3e05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:19.767914 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02629e4f-646e-4868-8698-bae6ed7c3e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:19.768454 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02629e4f-646e-4868-8698-bae6ed7c3e05 admin admin] PATH_INFO: `/v3/projects/430804382c704b7d9688bcbc8bfd0c08/users/f9256ac4bd704680bab981f8c2d3aa0c/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:19.772330 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344249e3-9489-4aaa-86ca-60063c5a3955 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:19.773834 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02629e4f-646e-4868-8698-bae6ed7c3e05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=430804382c704b7d9688bcbc8bfd0c08, user_id=f9256ac4bd704680bab981f8c2d3aa0c, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:19.777315 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02629e4f-646e-4868-8698-bae6ed7c3e05 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:19.786711 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c10ba-36f1-4c04-9191-1fbe5c0f3e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:19.787639 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c10ba-36f1-4c04-9191-1fbe5c0f3e4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:19.787928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c10ba-36f1-4c04-9191-1fbe5c0f3e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:19.788211 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c10ba-36f1-4c04-9191-1fbe5c0f3e4f admin admin] PATH_INFO: `/v3/users/09ec37c663074430951d659cb60a0023` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:19.789809 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c10ba-36f1-4c04-9191-1fbe5c0f3e4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=09ec37c663074430951d659cb60a0023)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:19.791835 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c10ba-36f1-4c04-9191-1fbe5c0f3e4f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:19.794879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:19.814316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 984/1953] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:19 2022] PUT /identity/v3/projects/430804382c704b7d9688bcbc8bfd0c08/users/f9256ac4bd704680bab981f8c2d3aa0c/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:35:19.816212 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d3d67-09c7-43d5-ad4f-1c88ae5a32aa None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:19.816511 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d3d67-09c7-43d5-ad4f-1c88ae5a32aa None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:19.816845 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d3d67-09c7-43d5-ad4f-1c88ae5a32aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:19.826778 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:19.877559 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-9e7d3d67-09c7-43d5-ad4f-1c88ae5a32aa None None] MFA Rules not processed for user `f9256ac4bd704680bab981f8c2d3aa0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:19.878662 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3d6c10ba-36f1-4c04-9191-1fbe5c0f3e4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09ec37c663074430951d659cb60a0023'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:19.878788 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-9e7d3d67-09c7-43d5-ad4f-1c88ae5a32aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:19.898762 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3d6c10ba-36f1-4c04-9191-1fbe5c0f3e4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09ec37c663074430951d659cb60a0023'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:19.900160 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3d6c10ba-36f1-4c04-9191-1fbe5c0f3e4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09ec37c663074430951d659cb60a0023'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:19.910086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3d6c10ba-36f1-4c04-9191-1fbe5c0f3e4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09ec37c663074430951d659cb60a0023'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:19.923446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3d6c10ba-36f1-4c04-9191-1fbe5c0f3e4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09ec37c663074430951d659cb60a0023'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:19.934919 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 970/1954] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:19 2022] DELETE /identity/v3/users/09ec37c663074430951d659cb60a0023 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:20.054651 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 985/1955] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:19 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:20.063494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c10ba-36f1-4c04-9191-1fbe5c0f3e4f admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:20.067215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3d6c10ba-36f1-4c04-9191-1fbe5c0f3e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:20.205063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c10ba-36f1-4c04-9191-1fbe5c0f3e4f admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:20.223095 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8a6b3-3925-459e-af4a-6b6022af4834 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:20.223889 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8a6b3-3925-459e-af4a-6b6022af4834 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:20.224257 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8a6b3-3925-459e-af4a-6b6022af4834 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:20.224609 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8a6b3-3925-459e-af4a-6b6022af4834 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:20.387437 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad8a6b3-3925-459e-af4a-6b6022af4834 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:20.392174 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad8a6b3-3925-459e-af4a-6b6022af4834 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:20.414281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 971/1956] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:20.472569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d3d67-09c7-43d5-ad4f-1c88ae5a32aa None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:20.474763 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-9e7d3d67-09c7-43d5-ad4f-1c88ae5a32aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:20.632539 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d3d67-09c7-43d5-ad4f-1c88ae5a32aa None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:20.657182 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f382ae-8ea3-429e-85f6-41712d34220a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:20.657874 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f382ae-8ea3-429e-85f6-41712d34220a service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:20.658165 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f382ae-8ea3-429e-85f6-41712d34220a service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:20.658450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f382ae-8ea3-429e-85f6-41712d34220a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:20.664201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-c6f382ae-8ea3-429e-85f6-41712d34220a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:20.801958 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f382ae-8ea3-429e-85f6-41712d34220a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:20.804896 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f382ae-8ea3-429e-85f6-41712d34220a service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:20.822156 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 986/1957] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:21.142065 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8a6b3-3925-459e-af4a-6b6022af4834 service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:21.144398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8ad8a6b3-3925-459e-af4a-6b6022af4834 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:21.278255 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:21.327482 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8a6b3-3925-459e-af4a-6b6022af4834 service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:21.339105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced73c39-8406-43d5-b2bc-645228ff43ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:21.340091 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced73c39-8406-43d5-b2bc-645228ff43ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:21.340442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced73c39-8406-43d5-b2bc-645228ff43ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:21.340727 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced73c39-8406-43d5-b2bc-645228ff43ee admin admin] PATH_INFO: `/v3/projects/7f74f72d8edd4bada30cb56b91a8dbbb` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:21.342693 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced73c39-8406-43d5-b2bc-645228ff43ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f74f72d8edd4bada30cb56b91a8dbbb)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:21.350188 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced73c39-8406-43d5-b2bc-645228ff43ee admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:21.487957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ced73c39-8406-43d5-b2bc-645228ff43ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f74f72d8edd4bada30cb56b91a8dbbb', 'request_id': 'req-ced73c39-8406-43d5-b2bc-645228ff43ee'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:21.519632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ced73c39-8406-43d5-b2bc-645228ff43ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f74f72d8edd4bada30cb56b91a8dbbb', 'request_id': 'req-ced73c39-8406-43d5-b2bc-645228ff43ee'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:21.530509 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ced73c39-8406-43d5-b2bc-645228ff43ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f74f72d8edd4bada30cb56b91a8dbbb', 'request_id': 'req-ced73c39-8406-43d5-b2bc-645228ff43ee'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:21.559293 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ced73c39-8406-43d5-b2bc-645228ff43ee admin admin] The token cache is being invalidate because project 7f74f72d8edd4bada30cb56b91a8dbbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:21.560017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ced73c39-8406-43d5-b2bc-645228ff43ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:21.560932 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ced73c39-8406-43d5-b2bc-645228ff43ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:21.567028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 972/1958] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:21 2022] DELETE /identity/v3/projects/7f74f72d8edd4bada30cb56b91a8dbbb => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:21.570501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef05667-229f-4eca-9a9c-573182d894cb None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:21.571024 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef05667-229f-4eca-9a9c-573182d894cb None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:21.571115 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef05667-229f-4eca-9a9c-573182d894cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:21.611955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-8ef05667-229f-4eca-9a9c-573182d894cb None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:21.613254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-8ef05667-229f-4eca-9a9c-573182d894cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:21.704552 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:21.757178 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 987/1959] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:21.767963 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e6d6e-1a60-4814-aad6-656e1f36a9a0 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:21.768171 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e6d6e-1a60-4814-aad6-656e1f36a9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:21.768435 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e6d6e-1a60-4814-aad6-656e1f36a9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:21.838282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-085e6d6e-1a60-4814-aad6-656e1f36a9a0 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:21.839534 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-085e6d6e-1a60-4814-aad6-656e1f36a9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:21.861689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 973/1960] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:21.867204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef05667-229f-4eca-9a9c-573182d894cb None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:21.883535 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef05667-229f-4eca-9a9c-573182d894cb None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:21.899296 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc91dd-d4cc-4628-8b54-9c299ed42a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:21.900194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc91dd-d4cc-4628-8b54-9c299ed42a53 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:21.900492 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc91dd-d4cc-4628-8b54-9c299ed42a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:21.900780 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc91dd-d4cc-4628-8b54-9c299ed42a53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:21.901572 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc91dd-d4cc-4628-8b54-9c299ed42a53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:35:21.901888 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc91dd-d4cc-4628-8b54-9c299ed42a53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:21.908074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc91dd-d4cc-4628-8b54-9c299ed42a53 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:21.939210 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 988/1961] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:35:21.951927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e6d6e-1a60-4814-aad6-656e1f36a9a0 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:21.968877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e6d6e-1a60-4814-aad6-656e1f36a9a0 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:21.985490 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6b2dd-04a8-479b-b575-e88d5903b6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:21.986374 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6b2dd-04a8-479b-b575-e88d5903b6e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:21.986748 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6b2dd-04a8-479b-b575-e88d5903b6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:21.987250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6b2dd-04a8-479b-b575-e88d5903b6e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:21.994659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac6b2dd-04a8-479b-b575-e88d5903b6e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:22.001812 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac6b2dd-04a8-479b-b575-e88d5903b6e2 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:22.054234 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 974/1962] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:21 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:22.054909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc91dd-d4cc-4628-8b54-9c299ed42a53 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:22.073844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc91dd-d4cc-4628-8b54-9c299ed42a53 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:22.163011 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e8b78-76ba-4066-94be-17a543afb265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:22.163924 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e8b78-76ba-4066-94be-17a543afb265 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:22.164496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e8b78-76ba-4066-94be-17a543afb265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:22.164881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e8b78-76ba-4066-94be-17a543afb265 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:22.165881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8e8b78-76ba-4066-94be-17a543afb265 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:22.173189 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8e8b78-76ba-4066-94be-17a543afb265 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:22.197721 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 989/1963] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:22 2022] POST /identity/v3/users => generated 286 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:22.202050 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6b2dd-04a8-479b-b575-e88d5903b6e2 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:22.223696 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6b2dd-04a8-479b-b575-e88d5903b6e2 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:22.251215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ac5be-7152-4d9d-b759-ddfa6e2bc8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:22.252005 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ac5be-7152-4d9d-b759-ddfa6e2bc8c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:22.252352 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ac5be-7152-4d9d-b759-ddfa6e2bc8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:22.252638 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ac5be-7152-4d9d-b759-ddfa6e2bc8c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:22.253631 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106ac5be-7152-4d9d-b759-ddfa6e2bc8c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:22.255332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106ac5be-7152-4d9d-b759-ddfa6e2bc8c8 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:22.271547 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 975/1964] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:22.277565 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e8b78-76ba-4066-94be-17a543afb265 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:22.296270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e8b78-76ba-4066-94be-17a543afb265 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:22.312099 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af56306-b3bf-44f3-ac7a-19a7600c0815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:22.312877 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af56306-b3bf-44f3-ac7a-19a7600c0815 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:22.313602 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af56306-b3bf-44f3-ac7a-19a7600c0815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:22.313971 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af56306-b3bf-44f3-ac7a-19a7600c0815 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:22.315565 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af56306-b3bf-44f3-ac7a-19a7600c0815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:22.318378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af56306-b3bf-44f3-ac7a-19a7600c0815 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:22.337350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 990/1965] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:22.341326 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ac5be-7152-4d9d-b759-ddfa6e2bc8c8 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:22.355603 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ac5be-7152-4d9d-b759-ddfa6e2bc8c8 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:22.360841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c49a116-25d3-470a-8cdf-0e9962bde717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:22.362101 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49a116-25d3-470a-8cdf-0e9962bde717 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:22.362401 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49a116-25d3-470a-8cdf-0e9962bde717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:22.363001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49a116-25d3-470a-8cdf-0e9962bde717 admin admin] PATH_INFO: `/v3/projects/9d6e1095a3c14792af1321afa0f40c89/users/eef5b7b727224077a1055b5fa26818ee/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:22.397555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c49a116-25d3-470a-8cdf-0e9962bde717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d6e1095a3c14792af1321afa0f40c89, user_id=eef5b7b727224077a1055b5fa26818ee, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:22.405023 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c49a116-25d3-470a-8cdf-0e9962bde717 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:22.436956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 976/1966] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:22 2022] PUT /identity/v3/projects/9d6e1095a3c14792af1321afa0f40c89/users/eef5b7b727224077a1055b5fa26818ee/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:22.447772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4671a-58ef-4fa4-9ae4-a412177e5cf4 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:22.448121 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4671a-58ef-4fa4-9ae4-a412177e5cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:22.448381 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4671a-58ef-4fa4-9ae4-a412177e5cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:22.509053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-10d4671a-58ef-4fa4-9ae4-a412177e5cf4 None None] MFA Rules not processed for user `eef5b7b727224077a1055b5fa26818ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:22.510449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-10d4671a-58ef-4fa4-9ae4-a412177e5cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:22.596907 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 991/1967] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:22.601286 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c49a116-25d3-470a-8cdf-0e9962bde717 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:22.687048 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c49a116-25d3-470a-8cdf-0e9962bde717 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:22.694315 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4ab63-1e4c-451d-a6ee-2ae4a71e9f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:22.695397 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4ab63-1e4c-451d-a6ee-2ae4a71e9f1b admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:22.695776 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4ab63-1e4c-451d-a6ee-2ae4a71e9f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:22.696136 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4ab63-1e4c-451d-a6ee-2ae4a71e9f1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:22.697079 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4ab63-1e4c-451d-a6ee-2ae4a71e9f1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:22.699123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4ab63-1e4c-451d-a6ee-2ae4a71e9f1b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:22.723179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:22.740789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 977/1968] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:22 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:35:22.742160 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d4671a-58ef-4fa4-9ae4-a412177e5cf4 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:22.766471 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d4671a-58ef-4fa4-9ae4-a412177e5cf4 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:22.814782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:22.859272 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57946d-6d4b-4088-8a3e-f74460a977c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:22.861801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57946d-6d4b-4088-8a3e-f74460a977c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:22.862440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57946d-6d4b-4088-8a3e-f74460a977c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:22.862985 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57946d-6d4b-4088-8a3e-f74460a977c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:22.864067 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57946d-6d4b-4088-8a3e-f74460a977c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:22.869688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57946d-6d4b-4088-8a3e-f74460a977c9 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:22.900261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 992/1969] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:22 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:22.903995 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4ab63-1e4c-451d-a6ee-2ae4a71e9f1b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:22.919537 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4ab63-1e4c-451d-a6ee-2ae4a71e9f1b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:22.926069 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9298b-024a-4dbf-a4ac-2c125adf5dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:22.926817 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9298b-024a-4dbf-a4ac-2c125adf5dca admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:22.927115 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9298b-024a-4dbf-a4ac-2c125adf5dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:22.927524 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9298b-024a-4dbf-a4ac-2c125adf5dca admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:22.929905 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9298b-024a-4dbf-a4ac-2c125adf5dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:22.931651 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9298b-024a-4dbf-a4ac-2c125adf5dca admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:22.949198 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 978/1970] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:22.951837 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57946d-6d4b-4088-8a3e-f74460a977c9 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:22.963160 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57946d-6d4b-4088-8a3e-f74460a977c9 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:22.969588 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1195022-8ed9-428a-a774-efc0563828cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:22.970735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1195022-8ed9-428a-a774-efc0563828cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:22.971358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1195022-8ed9-428a-a774-efc0563828cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:22.975233 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1195022-8ed9-428a-a774-efc0563828cb admin admin] PATH_INFO: `/v3/projects/04ebc234fdb2414fa96028d9e8e1026a/users/515d526ada694dba887a0eee45eb0926/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:23.000822 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1195022-8ed9-428a-a774-efc0563828cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=04ebc234fdb2414fa96028d9e8e1026a, user_id=515d526ada694dba887a0eee45eb0926, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:23.003765 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1195022-8ed9-428a-a774-efc0563828cb admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:23.029184 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 993/1971] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:22 2022] PUT /identity/v3/projects/04ebc234fdb2414fa96028d9e8e1026a/users/515d526ada694dba887a0eee45eb0926/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:23.034100 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9298b-024a-4dbf-a4ac-2c125adf5dca admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:23.050042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9298b-024a-4dbf-a4ac-2c125adf5dca admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:23.132004 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb6c0b-7702-4ff7-89af-c4f30b723d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:23.133821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb6c0b-7702-4ff7-89af-c4f30b723d23 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:23.134240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb6c0b-7702-4ff7-89af-c4f30b723d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:23.134861 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb6c0b-7702-4ff7-89af-c4f30b723d23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:23.136231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb6c0b-7702-4ff7-89af-c4f30b723d23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:23.140329 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb6c0b-7702-4ff7-89af-c4f30b723d23 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:23.157442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 979/1972] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:23.162007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1195022-8ed9-428a-a774-efc0563828cb admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:23.175502 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1195022-8ed9-428a-a774-efc0563828cb admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:23.187336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a96a14-14c3-4f82-9b87-d2fb6d906e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:23.199207 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a96a14-14c3-4f82-9b87-d2fb6d906e2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:23.199542 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a96a14-14c3-4f82-9b87-d2fb6d906e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:23.199884 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a96a14-14c3-4f82-9b87-d2fb6d906e2a admin admin] PATH_INFO: `/v3/projects/04ebc234fdb2414fa96028d9e8e1026a/users/515d526ada694dba887a0eee45eb0926/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:23.203068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a96a14-14c3-4f82-9b87-d2fb6d906e2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=04ebc234fdb2414fa96028d9e8e1026a, user_id=515d526ada694dba887a0eee45eb0926, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:23.205368 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a96a14-14c3-4f82-9b87-d2fb6d906e2a admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:23.222047 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 994/1973] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:23 2022] PUT /identity/v3/projects/04ebc234fdb2414fa96028d9e8e1026a/users/515d526ada694dba887a0eee45eb0926/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:23.234382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e7b83-515d-4bf6-8e35-cacb43203fb8 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:23.235294 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e7b83-515d-4bf6-8e35-cacb43203fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:23.235447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e7b83-515d-4bf6-8e35-cacb43203fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:23.271623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e03e7b83-515d-4bf6-8e35-cacb43203fb8 None None] MFA Rules not processed for user `515d526ada694dba887a0eee45eb0926`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:23.272880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e03e7b83-515d-4bf6-8e35-cacb43203fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:23.353425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 980/1974] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:23.366014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a96a14-14c3-4f82-9b87-d2fb6d906e2a admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:23.367727 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-06a96a14-14c3-4f82-9b87-d2fb6d906e2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:23.435178 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a96a14-14c3-4f82-9b87-d2fb6d906e2a admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:23.440338 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128cfdb-6032-49c7-8101-c67fa2cb5d9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:23.441033 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128cfdb-6032-49c7-8101-c67fa2cb5d9a service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:23.441408 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128cfdb-6032-49c7-8101-c67fa2cb5d9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:23.441693 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128cfdb-6032-49c7-8101-c67fa2cb5d9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:23.452585 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6128cfdb-6032-49c7-8101-c67fa2cb5d9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:23.452989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f1746-144e-4e73-8189-f88abd8f3bfb None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:23.453407 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f1746-144e-4e73-8189-f88abd8f3bfb None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:23.454869 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f1746-144e-4e73-8189-f88abd8f3bfb None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:23.454989 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6128cfdb-6032-49c7-8101-c67fa2cb5d9a service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:23.456603 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 981/1975] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 17:35:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:35:23.465952 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 995/1976] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:23 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:23.508759 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206abb5-3ed8-40e9-95e1-349f01ff3b0d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:23.509443 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206abb5-3ed8-40e9-95e1-349f01ff3b0d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:23.509975 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206abb5-3ed8-40e9-95e1-349f01ff3b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:23.546448 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-8206abb5-3ed8-40e9-95e1-349f01ff3b0d None None] MFA Rules not processed for user `75f0586a8c1941448d750918183f6dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:23.552828 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8206abb5-3ed8-40e9-95e1-349f01ff3b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:23.644940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 982/1977] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 17:35:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:35:23.682195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128cfdb-6032-49c7-8101-c67fa2cb5d9a service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:23.683635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-6128cfdb-6032-49c7-8101-c67fa2cb5d9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:23.712603 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128cfdb-6032-49c7-8101-c67fa2cb5d9a service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:23.723544 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2e5c4-6305-49ac-99c0-8186f48039b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75f0586a8c1941448d750918183f6dc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:23.724377 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2e5c4-6305-49ac-99c0-8186f48039b7 service placement] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:23.724860 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2e5c4-6305-49ac-99c0-8186f48039b7 service placement] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:23.725065 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2e5c4-6305-49ac-99c0-8186f48039b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:23.740955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2e5c4-6305-49ac-99c0-8186f48039b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:23.742567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2e5c4-6305-49ac-99c0-8186f48039b7 service placement] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:23.758294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 996/1978] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:24.029364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206abb5-3ed8-40e9-95e1-349f01ff3b0d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:24.034265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:24.098108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206abb5-3ed8-40e9-95e1-349f01ff3b0d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:24.103009 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8480e-b4f0-48d7-8240-4c85f812f66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:24.103926 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8480e-b4f0-48d7-8240-4c85f812f66b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:24.104217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8480e-b4f0-48d7-8240-4c85f812f66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:24.104499 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8480e-b4f0-48d7-8240-4c85f812f66b admin admin] PATH_INFO: `/v3/users/eef5b7b727224077a1055b5fa26818ee` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:24.105877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8480e-b4f0-48d7-8240-4c85f812f66b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eef5b7b727224077a1055b5fa26818ee)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:24.107887 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8480e-b4f0-48d7-8240-4c85f812f66b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:24.155716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a2c8480e-b4f0-48d7-8240-4c85f812f66b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eef5b7b727224077a1055b5fa26818ee'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:24.168566 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a2c8480e-b4f0-48d7-8240-4c85f812f66b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eef5b7b727224077a1055b5fa26818ee'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:24.169273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a2c8480e-b4f0-48d7-8240-4c85f812f66b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eef5b7b727224077a1055b5fa26818ee'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:24.176324 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a2c8480e-b4f0-48d7-8240-4c85f812f66b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eef5b7b727224077a1055b5fa26818ee'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:24.184167 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a2c8480e-b4f0-48d7-8240-4c85f812f66b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eef5b7b727224077a1055b5fa26818ee'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:24.191521 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 983/1979] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:24 2022] DELETE /identity/v3/users/eef5b7b727224077a1055b5fa26818ee => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:24.196297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587874d8-3b4b-4525-986b-0fe4170bc98e None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:24.196615 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587874d8-3b4b-4525-986b-0fe4170bc98e None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:24.196905 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587874d8-3b4b-4525-986b-0fe4170bc98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:24.218200 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-587874d8-3b4b-4525-986b-0fe4170bc98e None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:24.219667 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-587874d8-3b4b-4525-986b-0fe4170bc98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:24.278954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ab29a-19f9-4e5d-9367-9f1de58c574f None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:24.280045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ab29a-19f9-4e5d-9367-9f1de58c574f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:24.280594 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ab29a-19f9-4e5d-9367-9f1de58c574f None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:24.283947 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 984/1980] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 17:35:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:35:24.289200 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666ca01-6722-49ee-8082-c34d27f1a3ff None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:24.289575 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666ca01-6722-49ee-8082-c34d27f1a3ff None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:24.289938 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666ca01-6722-49ee-8082-c34d27f1a3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:24.291685 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 997/1981] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:35:24.302245 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587874d8-3b4b-4525-986b-0fe4170bc98e None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:24.303708 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-587874d8-3b4b-4525-986b-0fe4170bc98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:24.313048 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d666ca01-6722-49ee-8082-c34d27f1a3ff None None] MFA Rules not processed for user `75f0586a8c1941448d750918183f6dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:24.314390 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d666ca01-6722-49ee-8082-c34d27f1a3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:24.326607 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:24.391184 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587874d8-3b4b-4525-986b-0fe4170bc98e None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:24.395873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c25e94-10b6-47af-8ff2-cad506c12c0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:24.396539 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c25e94-10b6-47af-8ff2-cad506c12c0f service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:24.396830 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c25e94-10b6-47af-8ff2-cad506c12c0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:24.397178 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c25e94-10b6-47af-8ff2-cad506c12c0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:24.398963 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 985/1982] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 17:35:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:35:24.400952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666ca01-6722-49ee-8082-c34d27f1a3ff None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:24.402695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d666ca01-6722-49ee-8082-c34d27f1a3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:24.411072 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c25e94-10b6-47af-8ff2-cad506c12c0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:24.414206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c25e94-10b6-47af-8ff2-cad506c12c0f service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:24.423283 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666ca01-6722-49ee-8082-c34d27f1a3ff None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:24.426044 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 998/1983] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:24.426960 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c25e94-10b6-47af-8ff2-cad506c12c0f service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:24.428581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583a348-7d6a-4f8a-916a-bc15e4dc0547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:24.428878 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-83c25e94-10b6-47af-8ff2-cad506c12c0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:24.429651 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583a348-7d6a-4f8a-916a-bc15e4dc0547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:24.430019 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583a348-7d6a-4f8a-916a-bc15e4dc0547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:24.430375 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583a348-7d6a-4f8a-916a-bc15e4dc0547 admin admin] PATH_INFO: `/v3/users/c1ab33725f19417ebf15df836bf74ab9` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:24.431659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583a348-7d6a-4f8a-916a-bc15e4dc0547 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1ab33725f19417ebf15df836bf74ab9)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:24.436665 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583a348-7d6a-4f8a-916a-bc15e4dc0547 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:24.445359 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c25e94-10b6-47af-8ff2-cad506c12c0f service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:24.450840 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91919c00-129e-4f27-890e-327e6e19f115 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75f0586a8c1941448d750918183f6dc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:24.451494 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91919c00-129e-4f27-890e-327e6e19f115 service placement] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:24.451784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91919c00-129e-4f27-890e-327e6e19f115 service placement] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:24.452066 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91919c00-129e-4f27-890e-327e6e19f115 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:24.452529 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:24.462599 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91919c00-129e-4f27-890e-327e6e19f115 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:24.464278 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91919c00-129e-4f27-890e-327e6e19f115 service placement] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:24.477688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 999/1984] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:24.478851 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91919c00-129e-4f27-890e-327e6e19f115 service placement] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:24.480822 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-91919c00-129e-4f27-890e-327e6e19f115 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:24.499896 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91919c00-129e-4f27-890e-327e6e19f115 service placement] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:24.506700 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e5d50-833c-4dc2-b1ad-15de11116df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:24.507908 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e5d50-833c-4dc2-b1ad-15de11116df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:24.508346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e5d50-833c-4dc2-b1ad-15de11116df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:24.508749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e5d50-833c-4dc2-b1ad-15de11116df5 admin admin] PATH_INFO: `/v3/users/79fe74e3cab54be88f66756cea434356` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:24.510120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e5d50-833c-4dc2-b1ad-15de11116df5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79fe74e3cab54be88f66756cea434356)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:24.512298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e5d50-833c-4dc2-b1ad-15de11116df5 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:24.520749 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0583a348-7d6a-4f8a-916a-bc15e4dc0547 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1ab33725f19417ebf15df836bf74ab9'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:24.534732 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0583a348-7d6a-4f8a-916a-bc15e4dc0547 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1ab33725f19417ebf15df836bf74ab9'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:24.535615 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0583a348-7d6a-4f8a-916a-bc15e4dc0547 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1ab33725f19417ebf15df836bf74ab9'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:24.543308 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0583a348-7d6a-4f8a-916a-bc15e4dc0547 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1ab33725f19417ebf15df836bf74ab9'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:24.551551 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0583a348-7d6a-4f8a-916a-bc15e4dc0547 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1ab33725f19417ebf15df836bf74ab9'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:24.558945 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 986/1985] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:24 2022] DELETE /identity/v3/users/c1ab33725f19417ebf15df836bf74ab9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:24.577833 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c20e5d50-833c-4dc2-b1ad-15de11116df5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79fe74e3cab54be88f66756cea434356'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:24.590461 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c20e5d50-833c-4dc2-b1ad-15de11116df5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79fe74e3cab54be88f66756cea434356'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:24.591650 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c20e5d50-833c-4dc2-b1ad-15de11116df5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79fe74e3cab54be88f66756cea434356'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:24.604534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c20e5d50-833c-4dc2-b1ad-15de11116df5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79fe74e3cab54be88f66756cea434356'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:24.617631 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c20e5d50-833c-4dc2-b1ad-15de11116df5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79fe74e3cab54be88f66756cea434356'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:24.624014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1000/1986] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:24 2022] DELETE /identity/v3/users/79fe74e3cab54be88f66756cea434356 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:24.629199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1b3fd-f57f-4f74-9480-1ca363ae7e3a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:24.629499 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1b3fd-f57f-4f74-9480-1ca363ae7e3a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:24.629784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1b3fd-f57f-4f74-9480-1ca363ae7e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:24.666509 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-eeb1b3fd-f57f-4f74-9480-1ca363ae7e3a None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:24.669360 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-eeb1b3fd-f57f-4f74-9480-1ca363ae7e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:24.754359 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e5d50-833c-4dc2-b1ad-15de11116df5 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:24.756288 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-c20e5d50-833c-4dc2-b1ad-15de11116df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:24.759412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 987/1987] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:24.770318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1b3fd-f57f-4f74-9480-1ca363ae7e3a None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:24.771883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-eeb1b3fd-f57f-4f74-9480-1ca363ae7e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:24.773517 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e5d50-833c-4dc2-b1ad-15de11116df5 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:24.778441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02293975-e56f-4795-98d3-2ff36ae7d3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:24.779329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02293975-e56f-4795-98d3-2ff36ae7d3f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:24.779613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02293975-e56f-4795-98d3-2ff36ae7d3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:24.780070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02293975-e56f-4795-98d3-2ff36ae7d3f4 admin admin] PATH_INFO: `/v3/projects/9d6e1095a3c14792af1321afa0f40c89` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:24.781309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02293975-e56f-4795-98d3-2ff36ae7d3f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d6e1095a3c14792af1321afa0f40c89)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:24.784410 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02293975-e56f-4795-98d3-2ff36ae7d3f4 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:24.847086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1b3fd-f57f-4f74-9480-1ca363ae7e3a None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:24.850395 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-02293975-e56f-4795-98d3-2ff36ae7d3f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d6e1095a3c14792af1321afa0f40c89', 'request_id': 'req-02293975-e56f-4795-98d3-2ff36ae7d3f4'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:24.857569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-02293975-e56f-4795-98d3-2ff36ae7d3f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d6e1095a3c14792af1321afa0f40c89', 'request_id': 'req-02293975-e56f-4795-98d3-2ff36ae7d3f4'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:24.873413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-02293975-e56f-4795-98d3-2ff36ae7d3f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d6e1095a3c14792af1321afa0f40c89', 'request_id': 'req-02293975-e56f-4795-98d3-2ff36ae7d3f4'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:24.886236 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:24.890457 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-02293975-e56f-4795-98d3-2ff36ae7d3f4 admin admin] The token cache is being invalidate because project 9d6e1095a3c14792af1321afa0f40c89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:24.891059 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-02293975-e56f-4795-98d3-2ff36ae7d3f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:24.892276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-02293975-e56f-4795-98d3-2ff36ae7d3f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:24.894206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1001/1988] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:24 2022] DELETE /identity/v3/projects/9d6e1095a3c14792af1321afa0f40c89 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:24.898325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02293975-e56f-4795-98d3-2ff36ae7d3f4 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:24.900480 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-02293975-e56f-4795-98d3-2ff36ae7d3f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:24.904345 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:24.912487 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ddc5d-842b-431a-b0f0-e1ee66213cdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:24.913189 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ddc5d-842b-431a-b0f0-e1ee66213cdc service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:24.913641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ddc5d-842b-431a-b0f0-e1ee66213cdc service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:24.913925 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ddc5d-842b-431a-b0f0-e1ee66213cdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:24.916633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fc9ddc5d-842b-431a-b0f0-e1ee66213cdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:24.980191 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02293975-e56f-4795-98d3-2ff36ae7d3f4 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:24.987501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef325a-af76-4200-9812-ca378bda86d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:24.988521 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef325a-af76-4200-9812-ca378bda86d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:24.988904 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef325a-af76-4200-9812-ca378bda86d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:24.989319 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef325a-af76-4200-9812-ca378bda86d5 admin admin] PATH_INFO: `/v3/projects/c15148d9bba74dd0bb4cc6cc8960f1d6` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:24.991560 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef325a-af76-4200-9812-ca378bda86d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c15148d9bba74dd0bb4cc6cc8960f1d6)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:24.994247 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef325a-af76-4200-9812-ca378bda86d5 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:25.028026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ddc5d-842b-431a-b0f0-e1ee66213cdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:25.030770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ddc5d-842b-431a-b0f0-e1ee66213cdc service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:25.043619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 988/1989] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:25.044829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ddc5d-842b-431a-b0f0-e1ee66213cdc service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:25.046405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fc9ddc5d-842b-431a-b0f0-e1ee66213cdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:25.059378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-27ef325a-af76-4200-9812-ca378bda86d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c15148d9bba74dd0bb4cc6cc8960f1d6', 'request_id': 'req-27ef325a-af76-4200-9812-ca378bda86d5'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.061619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ddc5d-842b-431a-b0f0-e1ee66213cdc service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:25.067652 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-27ef325a-af76-4200-9812-ca378bda86d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c15148d9bba74dd0bb4cc6cc8960f1d6', 'request_id': 'req-27ef325a-af76-4200-9812-ca378bda86d5'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.075551 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-27ef325a-af76-4200-9812-ca378bda86d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c15148d9bba74dd0bb4cc6cc8960f1d6', 'request_id': 'req-27ef325a-af76-4200-9812-ca378bda86d5'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.105259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-27ef325a-af76-4200-9812-ca378bda86d5 admin admin] The token cache is being invalidate because project c15148d9bba74dd0bb4cc6cc8960f1d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:25.107114 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-27ef325a-af76-4200-9812-ca378bda86d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.108534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-27ef325a-af76-4200-9812-ca378bda86d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.110701 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1002/1990] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:24 2022] DELETE /identity/v3/projects/c15148d9bba74dd0bb4cc6cc8960f1d6 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:25.116754 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef325a-af76-4200-9812-ca378bda86d5 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:25.119766 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-27ef325a-af76-4200-9812-ca378bda86d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:25.147525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeead621-eec1-4373-bb5c-d75061f48f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:25.148647 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeead621-eec1-4373-bb5c-d75061f48f08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:25.148941 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeead621-eec1-4373-bb5c-d75061f48f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:25.149790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeead621-eec1-4373-bb5c-d75061f48f08 admin admin] PATH_INFO: `/v3/users/515d526ada694dba887a0eee45eb0926` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:25.151348 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeead621-eec1-4373-bb5c-d75061f48f08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=515d526ada694dba887a0eee45eb0926)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:25.154816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeead621-eec1-4373-bb5c-d75061f48f08 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:25.263449 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-aeead621-eec1-4373-bb5c-d75061f48f08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '515d526ada694dba887a0eee45eb0926'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.273431 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef325a-af76-4200-9812-ca378bda86d5 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:25.285526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a40b68-3d24-482b-af6a-c03341e9e806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:25.286440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a40b68-3d24-482b-af6a-c03341e9e806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:25.286737 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a40b68-3d24-482b-af6a-c03341e9e806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:25.287027 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a40b68-3d24-482b-af6a-c03341e9e806 admin admin] PATH_INFO: `/v3/users/7b72d74533104027958828ccda6af289` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:25.288388 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-aeead621-eec1-4373-bb5c-d75061f48f08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '515d526ada694dba887a0eee45eb0926'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.288907 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a40b68-3d24-482b-af6a-c03341e9e806 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b72d74533104027958828ccda6af289)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:25.289084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-aeead621-eec1-4373-bb5c-d75061f48f08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '515d526ada694dba887a0eee45eb0926'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.290875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a40b68-3d24-482b-af6a-c03341e9e806 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:25.300859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-aeead621-eec1-4373-bb5c-d75061f48f08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '515d526ada694dba887a0eee45eb0926'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.314737 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-aeead621-eec1-4373-bb5c-d75061f48f08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '515d526ada694dba887a0eee45eb0926'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.322928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 989/1991] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:25 2022] DELETE /identity/v3/users/515d526ada694dba887a0eee45eb0926 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:25.349409 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:25.373617 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-82a40b68-3d24-482b-af6a-c03341e9e806 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b72d74533104027958828ccda6af289'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.384992 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-82a40b68-3d24-482b-af6a-c03341e9e806 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b72d74533104027958828ccda6af289'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.385715 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-82a40b68-3d24-482b-af6a-c03341e9e806 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b72d74533104027958828ccda6af289'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.396982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-82a40b68-3d24-482b-af6a-c03341e9e806 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b72d74533104027958828ccda6af289'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.415632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-82a40b68-3d24-482b-af6a-c03341e9e806 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b72d74533104027958828ccda6af289'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.424608 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1003/1992] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:25 2022] DELETE /identity/v3/users/7b72d74533104027958828ccda6af289 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:25.450949 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeead621-eec1-4373-bb5c-d75061f48f08 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:25.452484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-aeead621-eec1-4373-bb5c-d75061f48f08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:25.475509 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:25.537068 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeead621-eec1-4373-bb5c-d75061f48f08 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:25.614012 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1711c9-e354-4c5c-aa45-58ca22fc2873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:25.615170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1711c9-e354-4c5c-aa45-58ca22fc2873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:25.615457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1711c9-e354-4c5c-aa45-58ca22fc2873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:25.615738 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1711c9-e354-4c5c-aa45-58ca22fc2873 admin admin] PATH_INFO: `/v3/projects/81bac1058fd248d897738ecba112fdf0` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:25.616941 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1711c9-e354-4c5c-aa45-58ca22fc2873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81bac1058fd248d897738ecba112fdf0)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:25.618920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1711c9-e354-4c5c-aa45-58ca22fc2873 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:25.689109 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a40b68-3d24-482b-af6a-c03341e9e806 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:25.690907 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-82a40b68-3d24-482b-af6a-c03341e9e806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:25.701112 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cb1711c9-e354-4c5c-aa45-58ca22fc2873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81bac1058fd248d897738ecba112fdf0', 'request_id': 'req-cb1711c9-e354-4c5c-aa45-58ca22fc2873'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.717636 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cb1711c9-e354-4c5c-aa45-58ca22fc2873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81bac1058fd248d897738ecba112fdf0', 'request_id': 'req-cb1711c9-e354-4c5c-aa45-58ca22fc2873'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.730580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a40b68-3d24-482b-af6a-c03341e9e806 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:25.732021 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cb1711c9-e354-4c5c-aa45-58ca22fc2873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81bac1058fd248d897738ecba112fdf0', 'request_id': 'req-cb1711c9-e354-4c5c-aa45-58ca22fc2873'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.752316 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cb1711c9-e354-4c5c-aa45-58ca22fc2873 admin admin] The token cache is being invalidate because project 81bac1058fd248d897738ecba112fdf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:25.752852 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cb1711c9-e354-4c5c-aa45-58ca22fc2873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.753826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cb1711c9-e354-4c5c-aa45-58ca22fc2873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.758894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 990/1993] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:25 2022] DELETE /identity/v3/projects/81bac1058fd248d897738ecba112fdf0 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:25.759856 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1711c9-e354-4c5c-aa45-58ca22fc2873 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:25.761284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-cb1711c9-e354-4c5c-aa45-58ca22fc2873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:25.799632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefdf00-2d4c-4077-bb25-dd06b76273ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:25.800595 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefdf00-2d4c-4077-bb25-dd06b76273ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:25.801220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefdf00-2d4c-4077-bb25-dd06b76273ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:25.801929 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefdf00-2d4c-4077-bb25-dd06b76273ae admin admin] PATH_INFO: `/v3/projects/04ebc234fdb2414fa96028d9e8e1026a` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:25.802856 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efefdf00-2d4c-4077-bb25-dd06b76273ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=04ebc234fdb2414fa96028d9e8e1026a)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:25.804767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efefdf00-2d4c-4077-bb25-dd06b76273ae admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:25.825874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:25.840180 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1711c9-e354-4c5c-aa45-58ca22fc2873 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:25.846333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2621e-222a-46d1-9c2c-312810fd7c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:25.847214 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2621e-222a-46d1-9c2c-312810fd7c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:25.847505 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2621e-222a-46d1-9c2c-312810fd7c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:25.847784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2621e-222a-46d1-9c2c-312810fd7c8c admin admin] PATH_INFO: `/v3/projects/2c4694d1b70a48608cffb580deef1741` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:25.849051 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2621e-222a-46d1-9c2c-312810fd7c8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c4694d1b70a48608cffb580deef1741)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:25.851258 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2621e-222a-46d1-9c2c-312810fd7c8c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:25.857220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-efefdf00-2d4c-4077-bb25-dd06b76273ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04ebc234fdb2414fa96028d9e8e1026a', 'request_id': 'req-efefdf00-2d4c-4077-bb25-dd06b76273ae'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.863487 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-efefdf00-2d4c-4077-bb25-dd06b76273ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04ebc234fdb2414fa96028d9e8e1026a', 'request_id': 'req-efefdf00-2d4c-4077-bb25-dd06b76273ae'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.869622 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-efefdf00-2d4c-4077-bb25-dd06b76273ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04ebc234fdb2414fa96028d9e8e1026a', 'request_id': 'req-efefdf00-2d4c-4077-bb25-dd06b76273ae'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.883146 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-efefdf00-2d4c-4077-bb25-dd06b76273ae admin admin] The token cache is being invalidate because project 04ebc234fdb2414fa96028d9e8e1026a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:25.883380 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-efefdf00-2d4c-4077-bb25-dd06b76273ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.884010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-efefdf00-2d4c-4077-bb25-dd06b76273ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.885605 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1004/1994] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:25 2022] DELETE /identity/v3/projects/04ebc234fdb2414fa96028d9e8e1026a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:25.889653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefdf00-2d4c-4077-bb25-dd06b76273ae admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:25.891004 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-efefdf00-2d4c-4077-bb25-dd06b76273ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:25.900298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-86c2621e-222a-46d1-9c2c-312810fd7c8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c4694d1b70a48608cffb580deef1741', 'request_id': 'req-86c2621e-222a-46d1-9c2c-312810fd7c8c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.908948 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-86c2621e-222a-46d1-9c2c-312810fd7c8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c4694d1b70a48608cffb580deef1741', 'request_id': 'req-86c2621e-222a-46d1-9c2c-312810fd7c8c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.916962 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-86c2621e-222a-46d1-9c2c-312810fd7c8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c4694d1b70a48608cffb580deef1741', 'request_id': 'req-86c2621e-222a-46d1-9c2c-312810fd7c8c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.929650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-86c2621e-222a-46d1-9c2c-312810fd7c8c admin admin] The token cache is being invalidate because project 2c4694d1b70a48608cffb580deef1741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:25.930059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-86c2621e-222a-46d1-9c2c-312810fd7c8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.930810 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-86c2621e-222a-46d1-9c2c-312810fd7c8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:25.932462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 991/1995] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:25 2022] DELETE /identity/v3/projects/2c4694d1b70a48608cffb580deef1741 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:25.936017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311fa72-c386-4eb9-9a0c-3185307ca0d1 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:25.936017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311fa72-c386-4eb9-9a0c-3185307ca0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:25.936017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311fa72-c386-4eb9-9a0c-3185307ca0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:25.939078 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:25.955917 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1311fa72-c386-4eb9-9a0c-3185307ca0d1 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:25.957127 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1311fa72-c386-4eb9-9a0c-3185307ca0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:25.963726 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefdf00-2d4c-4077-bb25-dd06b76273ae admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:26.018231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 992/1996] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:26.019796 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e522baa-403e-4d21-9cf0-bd4d31109dc4 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.020090 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e522baa-403e-4d21-9cf0-bd4d31109dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.020384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e522baa-403e-4d21-9cf0-bd4d31109dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.027359 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5e2ce-25af-4c9a-86a9-5ad4ff9f5cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:26.028247 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5e2ce-25af-4c9a-86a9-5ad4ff9f5cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.028534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5e2ce-25af-4c9a-86a9-5ad4ff9f5cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.028813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5e2ce-25af-4c9a-86a9-5ad4ff9f5cd7 admin admin] PATH_INFO: `/v3/users/97722ed40cc4451d81ef267b18dbb51a` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.030107 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd5e2ce-25af-4c9a-86a9-5ad4ff9f5cd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97722ed40cc4451d81ef267b18dbb51a)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:26.033113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd5e2ce-25af-4c9a-86a9-5ad4ff9f5cd7 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:26.043108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0e522baa-403e-4d21-9cf0-bd4d31109dc4 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:26.044326 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0e522baa-403e-4d21-9cf0-bd4d31109dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:26.053364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 993/1997] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:26.054534 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b68ad0-f0e1-4b84-b465-8c6303c011a2 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.054820 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b68ad0-f0e1-4b84-b465-8c6303c011a2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.055109 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b68ad0-f0e1-4b84-b465-8c6303c011a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.075882 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d3b68ad0-f0e1-4b84-b465-8c6303c011a2 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:26.077088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d3b68ad0-f0e1-4b84-b465-8c6303c011a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:26.082839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-1cd5e2ce-25af-4c9a-86a9-5ad4ff9f5cd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97722ed40cc4451d81ef267b18dbb51a'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:26.086460 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 994/1998] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:26.087977 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2ca27-f828-43bb-9d47-db23dc485040 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.088339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2ca27-f828-43bb-9d47-db23dc485040 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.088691 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2ca27-f828-43bb-9d47-db23dc485040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.095548 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-1cd5e2ce-25af-4c9a-86a9-5ad4ff9f5cd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97722ed40cc4451d81ef267b18dbb51a'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:26.096417 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-1cd5e2ce-25af-4c9a-86a9-5ad4ff9f5cd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97722ed40cc4451d81ef267b18dbb51a'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:26.106249 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-1cd5e2ce-25af-4c9a-86a9-5ad4ff9f5cd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97722ed40cc4451d81ef267b18dbb51a'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:26.108957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-dab2ca27-f828-43bb-9d47-db23dc485040 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:26.110814 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-dab2ca27-f828-43bb-9d47-db23dc485040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:26.122795 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 995/1999] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:26.122795 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2ca27-f828-43bb-9d47-db23dc485040 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:26.126658 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-1cd5e2ce-25af-4c9a-86a9-5ad4ff9f5cd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97722ed40cc4451d81ef267b18dbb51a'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:26.133092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2ca27-f828-43bb-9d47-db23dc485040 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:26.134137 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1005/2000] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:25 2022] DELETE /identity/v3/users/97722ed40cc4451d81ef267b18dbb51a => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:26.135093 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5e2ce-25af-4c9a-86a9-5ad4ff9f5cd7 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:26.136540 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-1cd5e2ce-25af-4c9a-86a9-5ad4ff9f5cd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:26.212717 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b7f440-b186-4d49-98bb-c1edb8798599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:26.214118 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7f440-b186-4d49-98bb-c1edb8798599 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.215166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7f440-b186-4d49-98bb-c1edb8798599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.215464 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7f440-b186-4d49-98bb-c1edb8798599 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.216227 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b7f440-b186-4d49-98bb-c1edb8798599 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:35:26.216540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b7f440-b186-4d49-98bb-c1edb8798599 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:26.218675 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b7f440-b186-4d49-98bb-c1edb8798599 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:26.228513 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5e2ce-25af-4c9a-86a9-5ad4ff9f5cd7 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:26.229930 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:26.236646 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3576cf-9656-40cd-9b6e-f4a31c269f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:26.237430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3576cf-9656-40cd-9b6e-f4a31c269f12 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.237886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3576cf-9656-40cd-9b6e-f4a31c269f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.238313 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3576cf-9656-40cd-9b6e-f4a31c269f12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.239240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3576cf-9656-40cd-9b6e-f4a31c269f12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:35:26.239572 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3576cf-9656-40cd-9b6e-f4a31c269f12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:26.241356 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3576cf-9656-40cd-9b6e-f4a31c269f12 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:26.247628 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 996/2001] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:35:26.248787 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b7f440-b186-4d49-98bb-c1edb8798599 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:26.252732 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e2b7f440-b186-4d49-98bb-c1edb8798599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:26.255764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:26.261785 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1006/2002] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:35:26.268945 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3576cf-9656-40cd-9b6e-f4a31c269f12 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:26.270195 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b7f440-b186-4d49-98bb-c1edb8798599 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:26.276263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e0fff-69ac-4527-81b2-af645bf85296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:26.277776 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e0fff-69ac-4527-81b2-af645bf85296 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.278191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e0fff-69ac-4527-81b2-af645bf85296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.278407 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e0fff-69ac-4527-81b2-af645bf85296 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.279119 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e0fff-69ac-4527-81b2-af645bf85296 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:26.281799 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e0fff-69ac-4527-81b2-af645bf85296 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:26.283741 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3576cf-9656-40cd-9b6e-f4a31c269f12 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:26.290903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fadf5e7-aed7-41be-bd49-e52215d6dbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:26.291951 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fadf5e7-aed7-41be-bd49-e52215d6dbdc admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.292479 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fadf5e7-aed7-41be-bd49-e52215d6dbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.293020 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fadf5e7-aed7-41be-bd49-e52215d6dbdc admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.294039 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fadf5e7-aed7-41be-bd49-e52215d6dbdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:26.296323 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fadf5e7-aed7-41be-bd49-e52215d6dbdc admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:26.313397 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 997/2003] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:26 2022] POST /identity/v3/projects => generated 367 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:26.317844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e0fff-69ac-4527-81b2-af645bf85296 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:26.337836 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1007/2004] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:26 2022] POST /identity/v3/projects => generated 369 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:26.338912 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fadf5e7-aed7-41be-bd49-e52215d6dbdc admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:26.388652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e0fff-69ac-4527-81b2-af645bf85296 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:26.428200 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fadf5e7-aed7-41be-bd49-e52215d6dbdc admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:26.433489 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca7bf6-a7e8-43e8-bc08-c9f524199f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:26.434358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca7bf6-a7e8-43e8-bc08-c9f524199f85 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.434656 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca7bf6-a7e8-43e8-bc08-c9f524199f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.435122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca7bf6-a7e8-43e8-bc08-c9f524199f85 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.435826 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca7bf6-a7e8-43e8-bc08-c9f524199f85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:26.438955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca7bf6-a7e8-43e8-bc08-c9f524199f85 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:26.457291 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2ff45-fcf8-497b-bc9a-a91778592bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:26.457953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2ff45-fcf8-497b-bc9a-a91778592bc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.458421 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2ff45-fcf8-497b-bc9a-a91778592bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.458664 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2ff45-fcf8-497b-bc9a-a91778592bc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.459444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2ff45-fcf8-497b-bc9a-a91778592bc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:26.461470 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2ff45-fcf8-497b-bc9a-a91778592bc2 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:26.465816 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1008/2005] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:26 2022] POST /identity/v3/users => generated 285 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:26.470331 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca7bf6-a7e8-43e8-bc08-c9f524199f85 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:26.482983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca7bf6-a7e8-43e8-bc08-c9f524199f85 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:26.491213 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f823d-eee3-430b-93cd-5a59ca9d8b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:26.491904 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f823d-eee3-430b-93cd-5a59ca9d8b71 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.492193 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f823d-eee3-430b-93cd-5a59ca9d8b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.492476 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f823d-eee3-430b-93cd-5a59ca9d8b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.493240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008f823d-eee3-430b-93cd-5a59ca9d8b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:26.495402 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008f823d-eee3-430b-93cd-5a59ca9d8b71 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:26.503709 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 998/2006] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:26 2022] POST /identity/v3/users => generated 284 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:35:26.504664 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2ff45-fcf8-497b-bc9a-a91778592bc2 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:26.512050 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1009/2007] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:26.518847 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f823d-eee3-430b-93cd-5a59ca9d8b71 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:26.520121 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2ff45-fcf8-497b-bc9a-a91778592bc2 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:26.526922 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c2c82-f632-4bb4-99ad-780455f4d1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:26.527705 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c2c82-f632-4bb4-99ad-780455f4d1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.528308 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c2c82-f632-4bb4-99ad-780455f4d1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.528674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c2c82-f632-4bb4-99ad-780455f4d1d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.529518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c2c82-f632-4bb4-99ad-780455f4d1d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:26.532054 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c2c82-f632-4bb4-99ad-780455f4d1d1 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:26.536484 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f823d-eee3-430b-93cd-5a59ca9d8b71 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:26.542284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d325919-ed90-4d26-99d7-2b092d754cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:26.543526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d325919-ed90-4d26-99d7-2b092d754cab admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.544006 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d325919-ed90-4d26-99d7-2b092d754cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.544510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d325919-ed90-4d26-99d7-2b092d754cab admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.544618 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 999/2008] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:26.545632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d325919-ed90-4d26-99d7-2b092d754cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:26.548184 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d325919-ed90-4d26-99d7-2b092d754cab admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:26.549259 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c2c82-f632-4bb4-99ad-780455f4d1d1 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:26.559200 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c2c82-f632-4bb4-99ad-780455f4d1d1 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:26.565058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1010/2009] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:26.566788 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d325919-ed90-4d26-99d7-2b092d754cab admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:26.567239 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002b777-d66a-466d-bc50-78b701194993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:26.567937 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002b777-d66a-466d-bc50-78b701194993 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.568226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002b777-d66a-466d-bc50-78b701194993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.568506 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002b777-d66a-466d-bc50-78b701194993 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.569282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002b777-d66a-466d-bc50-78b701194993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:26.570882 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002b777-d66a-466d-bc50-78b701194993 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:26.578581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d325919-ed90-4d26-99d7-2b092d754cab admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:26.583015 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1000/2010] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:35:26.586853 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002b777-d66a-466d-bc50-78b701194993 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:26.587599 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c335ab-1b3e-4076-835c-70f437605d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:26.588117 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c335ab-1b3e-4076-835c-70f437605d5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.588399 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c335ab-1b3e-4076-835c-70f437605d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.588683 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c335ab-1b3e-4076-835c-70f437605d5b admin admin] PATH_INFO: `/v3/projects/4e0539eb35ce48819c617525753a27bd/users/f401650914a7409e81287b48c1f3ebca/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.598478 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002b777-d66a-466d-bc50-78b701194993 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:26.604221 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d020d-a54d-445f-805d-99e0dca1a434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:26.604472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d020d-a54d-445f-805d-99e0dca1a434 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.605191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d020d-a54d-445f-805d-99e0dca1a434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.605484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d020d-a54d-445f-805d-99e0dca1a434 admin admin] PATH_INFO: `/v3/projects/72f7d001be1744bfbefb363a5c296497/users/0ebc99db6ab74668bd3d09ec6b89f75d/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.612287 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c335ab-1b3e-4076-835c-70f437605d5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e0539eb35ce48819c617525753a27bd, user_id=f401650914a7409e81287b48c1f3ebca, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:26.615662 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c335ab-1b3e-4076-835c-70f437605d5b admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:26.626126 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d020d-a54d-445f-805d-99e0dca1a434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72f7d001be1744bfbefb363a5c296497, user_id=0ebc99db6ab74668bd3d09ec6b89f75d, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:26.629380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d020d-a54d-445f-805d-99e0dca1a434 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:26.638421 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1011/2011] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:26 2022] PUT /identity/v3/projects/4e0539eb35ce48819c617525753a27bd/users/f401650914a7409e81287b48c1f3ebca/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:26.639358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c335ab-1b3e-4076-835c-70f437605d5b admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:26.651816 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-43c335ab-1b3e-4076-835c-70f437605d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:26.661777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1001/2012] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:26 2022] PUT /identity/v3/projects/72f7d001be1744bfbefb363a5c296497/users/0ebc99db6ab74668bd3d09ec6b89f75d/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:35:26.663400 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddac00-b89f-4c80-ac89-1b97bc8d96cb None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.663849 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddac00-b89f-4c80-ac89-1b97bc8d96cb None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.664321 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddac00-b89f-4c80-ac89-1b97bc8d96cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.694609 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-c6ddac00-b89f-4c80-ac89-1b97bc8d96cb None None] MFA Rules not processed for user `f401650914a7409e81287b48c1f3ebca`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:26.695881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c6ddac00-b89f-4c80-ac89-1b97bc8d96cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:26.710190 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:26.738072 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c335ab-1b3e-4076-835c-70f437605d5b admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:26.765730 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:26.768063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1002/2013] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:26.769797 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8727d63-a161-43b1-85c3-466b0f6d2dcf None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.770157 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8727d63-a161-43b1-85c3-466b0f6d2dcf None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.770439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8727d63-a161-43b1-85c3-466b0f6d2dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.806106 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e8727d63-a161-43b1-85c3-466b0f6d2dcf None None] MFA Rules not processed for user `0ebc99db6ab74668bd3d09ec6b89f75d`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:26.806444 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa2d3c-9b18-41f1-a14b-3748764dc1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:26.807343 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa2d3c-9b18-41f1-a14b-3748764dc1fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.807635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa2d3c-9b18-41f1-a14b-3748764dc1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.807919 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa2d3c-9b18-41f1-a14b-3748764dc1fb admin admin] PATH_INFO: `/v3/projects/88dbc9ed2dbb4ed1950a68407f014d35` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.809189 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa2d3c-9b18-41f1-a14b-3748764dc1fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=88dbc9ed2dbb4ed1950a68407f014d35)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:26.810073 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e8727d63-a161-43b1-85c3-466b0f6d2dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:26.811326 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa2d3c-9b18-41f1-a14b-3748764dc1fb admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:26.868289 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-43aa2d3c-9b18-41f1-a14b-3748764dc1fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88dbc9ed2dbb4ed1950a68407f014d35', 'request_id': 'req-43aa2d3c-9b18-41f1-a14b-3748764dc1fb'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:26.875471 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-43aa2d3c-9b18-41f1-a14b-3748764dc1fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88dbc9ed2dbb4ed1950a68407f014d35', 'request_id': 'req-43aa2d3c-9b18-41f1-a14b-3748764dc1fb'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:26.883414 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-43aa2d3c-9b18-41f1-a14b-3748764dc1fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88dbc9ed2dbb4ed1950a68407f014d35', 'request_id': 'req-43aa2d3c-9b18-41f1-a14b-3748764dc1fb'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:26.903859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-43aa2d3c-9b18-41f1-a14b-3748764dc1fb admin admin] The token cache is being invalidate because project 88dbc9ed2dbb4ed1950a68407f014d35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:26.904384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-43aa2d3c-9b18-41f1-a14b-3748764dc1fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:26.905495 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-43aa2d3c-9b18-41f1-a14b-3748764dc1fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:26.908196 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1012/2014] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:26 2022] DELETE /identity/v3/projects/88dbc9ed2dbb4ed1950a68407f014d35 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:26.909180 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa2d3c-9b18-41f1-a14b-3748764dc1fb admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:26.911378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-43aa2d3c-9b18-41f1-a14b-3748764dc1fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:26.913686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1003/2015] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:26 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:26.914625 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8727d63-a161-43b1-85c3-466b0f6d2dcf None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:26.930015 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8727d63-a161-43b1-85c3-466b0f6d2dcf None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:26.994332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa2d3c-9b18-41f1-a14b-3748764dc1fb admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:26.999054 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f4f6a-b3a0-4bdb-96a5-710153958c4a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327b347d6295422c99ce7b44eab00467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:27.000928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f4f6a-b3a0-4bdb-96a5-710153958c4a service glance] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:27.001340 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f4f6a-b3a0-4bdb-96a5-710153958c4a service glance] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:27.001634 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f4f6a-b3a0-4bdb-96a5-710153958c4a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:27.005230 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-fb8f4f6a-b3a0-4bdb-96a5-710153958c4a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:27.025903 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f76e42-53ce-4359-a33c-5efd05f01497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:27.027284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f76e42-53ce-4359-a33c-5efd05f01497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:27.027284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f76e42-53ce-4359-a33c-5efd05f01497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:27.027534 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f76e42-53ce-4359-a33c-5efd05f01497 admin admin] PATH_INFO: `/v3/users/f9256ac4bd704680bab981f8c2d3aa0c` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:27.028759 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f76e42-53ce-4359-a33c-5efd05f01497 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9256ac4bd704680bab981f8c2d3aa0c)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:27.030880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f76e42-53ce-4359-a33c-5efd05f01497 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:27.081472 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f4f6a-b3a0-4bdb-96a5-710153958c4a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:27.083264 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f4f6a-b3a0-4bdb-96a5-710153958c4a service glance] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:27.095895 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1013/2016] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:26 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:27.096889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f4f6a-b3a0-4bdb-96a5-710153958c4a service glance] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:27.099102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-fb8f4f6a-b3a0-4bdb-96a5-710153958c4a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:27.105284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-09f76e42-53ce-4359-a33c-5efd05f01497 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9256ac4bd704680bab981f8c2d3aa0c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:27.106814 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:27.116561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f4f6a-b3a0-4bdb-96a5-710153958c4a service glance] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:27.125042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b0035-6d97-4d46-b24f-6fb829b05ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:27.127014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b0035-6d97-4d46-b24f-6fb829b05ce4 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:27.127492 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b0035-6d97-4d46-b24f-6fb829b05ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:27.127963 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b0035-6d97-4d46-b24f-6fb829b05ce4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:27.128854 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b0035-6d97-4d46-b24f-6fb829b05ce4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:27.131165 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b0035-6d97-4d46-b24f-6fb829b05ce4 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:27.137549 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-09f76e42-53ce-4359-a33c-5efd05f01497 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9256ac4bd704680bab981f8c2d3aa0c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:27.138362 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-09f76e42-53ce-4359-a33c-5efd05f01497 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9256ac4bd704680bab981f8c2d3aa0c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:27.149031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:27.152336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-09f76e42-53ce-4359-a33c-5efd05f01497 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9256ac4bd704680bab981f8c2d3aa0c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:27.163644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-09f76e42-53ce-4359-a33c-5efd05f01497 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9256ac4bd704680bab981f8c2d3aa0c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:27.170968 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1004/2017] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:26 2022] DELETE /identity/v3/users/f9256ac4bd704680bab981f8c2d3aa0c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:27.171935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1014/2018] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:27 2022] POST /identity/v3/projects => generated 369 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:27.174781 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f76e42-53ce-4359-a33c-5efd05f01497 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:27.176330 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-09f76e42-53ce-4359-a33c-5efd05f01497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:27.274889 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f76e42-53ce-4359-a33c-5efd05f01497 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:27.280337 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c251a-2ef8-48a2-89fa-394449935c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:27.280835 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c251a-2ef8-48a2-89fa-394449935c52 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:27.281131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c251a-2ef8-48a2-89fa-394449935c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:27.282013 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c251a-2ef8-48a2-89fa-394449935c52 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:27.282766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c251a-2ef8-48a2-89fa-394449935c52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:27.284721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c251a-2ef8-48a2-89fa-394449935c52 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:27.308478 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1005/2019] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:27 2022] POST /identity/v3/users => generated 291 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:27.313072 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b0035-6d97-4d46-b24f-6fb829b05ce4 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:27.393249 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b0035-6d97-4d46-b24f-6fb829b05ce4 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:27.399632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c14bd-1cc2-4ebe-aba9-5ffa2ee3cdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:27.400331 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c14bd-1cc2-4ebe-aba9-5ffa2ee3cdc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:27.400686 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c14bd-1cc2-4ebe-aba9-5ffa2ee3cdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:27.400970 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c14bd-1cc2-4ebe-aba9-5ffa2ee3cdc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:27.401825 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c14bd-1cc2-4ebe-aba9-5ffa2ee3cdc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:27.403456 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c14bd-1cc2-4ebe-aba9-5ffa2ee3cdc1 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:27.420743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1015/2020] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:35:27.421895 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c251a-2ef8-48a2-89fa-394449935c52 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:27.432944 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c251a-2ef8-48a2-89fa-394449935c52 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:27.440023 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128000b-a531-4690-99c4-96aa35ff7bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:27.441474 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128000b-a531-4690-99c4-96aa35ff7bd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:27.442139 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128000b-a531-4690-99c4-96aa35ff7bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:27.442529 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128000b-a531-4690-99c4-96aa35ff7bd1 admin admin] PATH_INFO: `/v3/projects/4d5c976ab6d44f8f9906bdcac7a8bfe7/users/31bc434740004363a869456f7360d567/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:27.465837 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128000b-a531-4690-99c4-96aa35ff7bd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d5c976ab6d44f8f9906bdcac7a8bfe7, user_id=31bc434740004363a869456f7360d567, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:27.469883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128000b-a531-4690-99c4-96aa35ff7bd1 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:27.491192 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1006/2021] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:27 2022] PUT /identity/v3/projects/4d5c976ab6d44f8f9906bdcac7a8bfe7/users/31bc434740004363a869456f7360d567/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:27.496919 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c14bd-1cc2-4ebe-aba9-5ffa2ee3cdc1 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:27.515528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c14bd-1cc2-4ebe-aba9-5ffa2ee3cdc1 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:27.599947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03c243-67b2-427c-a762-b8f40050fc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:27.600911 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03c243-67b2-427c-a762-b8f40050fc3e admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:27.601242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03c243-67b2-427c-a762-b8f40050fc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:27.601957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03c243-67b2-427c-a762-b8f40050fc3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:27.602551 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03c243-67b2-427c-a762-b8f40050fc3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:27.604319 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03c243-67b2-427c-a762-b8f40050fc3e admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:27.616476 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1016/2022] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:27.620589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128000b-a531-4690-99c4-96aa35ff7bd1 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:27.634093 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128000b-a531-4690-99c4-96aa35ff7bd1 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:27.639823 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd0cec-0f57-4022-b3e5-0bba545e9630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:27.640691 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd0cec-0f57-4022-b3e5-0bba545e9630 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:27.641234 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd0cec-0f57-4022-b3e5-0bba545e9630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:27.643493 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd0cec-0f57-4022-b3e5-0bba545e9630 admin admin] PATH_INFO: `/v3/projects/4d5c976ab6d44f8f9906bdcac7a8bfe7/users/31bc434740004363a869456f7360d567/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:27.644834 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd0cec-0f57-4022-b3e5-0bba545e9630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d5c976ab6d44f8f9906bdcac7a8bfe7, user_id=31bc434740004363a869456f7360d567, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:27.647789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd0cec-0f57-4022-b3e5-0bba545e9630 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:27.677582 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1007/2023] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:27 2022] PUT /identity/v3/projects/4d5c976ab6d44f8f9906bdcac7a8bfe7/users/31bc434740004363a869456f7360d567/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:27.687235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe2149-b66a-405c-95e2-42f2b46678a1 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:27.687656 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe2149-b66a-405c-95e2-42f2b46678a1 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:27.688611 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe2149-b66a-405c-95e2-42f2b46678a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:27.734394 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-1fbe2149-b66a-405c-95e2-42f2b46678a1 None None] MFA Rules not processed for user `31bc434740004363a869456f7360d567`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:27.737298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-1fbe2149-b66a-405c-95e2-42f2b46678a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:27.793832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd0cec-0f57-4022-b3e5-0bba545e9630 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:27.795848 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c6dd0cec-0f57-4022-b3e5-0bba545e9630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:27.889653 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd0cec-0f57-4022-b3e5-0bba545e9630 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:27.895037 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1017/2024] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:27 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:27.897904 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e693a20-8c41-48fa-af9e-da17ff291ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:27.899278 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e693a20-8c41-48fa-af9e-da17ff291ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:27.903867 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e693a20-8c41-48fa-af9e-da17ff291ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:27.904268 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e693a20-8c41-48fa-af9e-da17ff291ce4 admin admin] PATH_INFO: `/v3/projects/430804382c704b7d9688bcbc8bfd0c08` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:27.907188 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e693a20-8c41-48fa-af9e-da17ff291ce4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=430804382c704b7d9688bcbc8bfd0c08)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:27.907623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe2149-b66a-405c-95e2-42f2b46678a1 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:27.909071 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-1fbe2149-b66a-405c-95e2-42f2b46678a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:27.911141 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e693a20-8c41-48fa-af9e-da17ff291ce4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:27.920704 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:27.946438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=91,keystone:DELETE=3,keystone:INSERT=6 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:27.992613 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8e693a20-8c41-48fa-af9e-da17ff291ce4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '430804382c704b7d9688bcbc8bfd0c08', 'request_id': 'req-8e693a20-8c41-48fa-af9e-da17ff291ce4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:28.005222 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe2149-b66a-405c-95e2-42f2b46678a1 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:28.012176 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8e693a20-8c41-48fa-af9e-da17ff291ce4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '430804382c704b7d9688bcbc8bfd0c08', 'request_id': 'req-8e693a20-8c41-48fa-af9e-da17ff291ce4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:28.021006 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8e693a20-8c41-48fa-af9e-da17ff291ce4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '430804382c704b7d9688bcbc8bfd0c08', 'request_id': 'req-8e693a20-8c41-48fa-af9e-da17ff291ce4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:28.036433 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8e693a20-8c41-48fa-af9e-da17ff291ce4 admin admin] The token cache is being invalidate because project 430804382c704b7d9688bcbc8bfd0c08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:28.036913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8e693a20-8c41-48fa-af9e-da17ff291ce4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:28.037750 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8e693a20-8c41-48fa-af9e-da17ff291ce4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:28.040046 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1008/2025] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:27 2022] DELETE /identity/v3/projects/430804382c704b7d9688bcbc8bfd0c08 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:35:28.045916 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d3e9e-8d9b-4c99-b560-e7e7f609eebc None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:28.046359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d3e9e-8d9b-4c99-b560-e7e7f609eebc None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:28.046691 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d3e9e-8d9b-4c99-b560-e7e7f609eebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:28.087072 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8e776-24c5-4bc0-8e3b-13aaf6115e9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:28.088528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8e776-24c5-4bc0-8e3b-13aaf6115e9d service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:28.088946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8e776-24c5-4bc0-8e3b-13aaf6115e9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:28.089366 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8e776-24c5-4bc0-8e3b-13aaf6115e9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:28.092558 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-17e8e776-24c5-4bc0-8e3b-13aaf6115e9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:28.093432 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-673d3e9e-8d9b-4c99-b560-e7e7f609eebc None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:28.096756 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-673d3e9e-8d9b-4c99-b560-e7e7f609eebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:28.181820 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8e776-24c5-4bc0-8e3b-13aaf6115e9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:28.183956 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8e776-24c5-4bc0-8e3b-13aaf6115e9d service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:28.193276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1009/2026] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:28.198257 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1018/2027] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:27 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:35:28.207926 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105839b-e36d-42ce-add1-d3104e279a6d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:28.208298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105839b-e36d-42ce-add1-d3104e279a6d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:28.208662 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105839b-e36d-42ce-add1-d3104e279a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:28.256356 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-6105839b-e36d-42ce-add1-d3104e279a6d None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:28.258174 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6105839b-e36d-42ce-add1-d3104e279a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:28.278839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1010/2028] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:28.284766 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8e776-24c5-4bc0-8e3b-13aaf6115e9d service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:28.299583 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8e776-24c5-4bc0-8e3b-13aaf6115e9d service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:28.320795 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba35dc-0923-445d-88d6-7b2eaf96b9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:28.322597 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba35dc-0923-445d-88d6-7b2eaf96b9ba admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:28.322934 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba35dc-0923-445d-88d6-7b2eaf96b9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:28.326486 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba35dc-0923-445d-88d6-7b2eaf96b9ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:28.326486 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba35dc-0923-445d-88d6-7b2eaf96b9ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:35:28.326486 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba35dc-0923-445d-88d6-7b2eaf96b9ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:28.327688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba35dc-0923-445d-88d6-7b2eaf96b9ba admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:28.365104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1019/2029] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:35:28.371723 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6105839b-e36d-42ce-add1-d3104e279a6d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:28.388565 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6105839b-e36d-42ce-add1-d3104e279a6d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:28.405871 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f718e3-b983-4f95-84ab-ce1bed4666f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:28.405871 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f718e3-b983-4f95-84ab-ce1bed4666f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:28.405871 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f718e3-b983-4f95-84ab-ce1bed4666f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:28.405871 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f718e3-b983-4f95-84ab-ce1bed4666f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:28.406702 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f718e3-b983-4f95-84ab-ce1bed4666f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:28.406702 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f718e3-b983-4f95-84ab-ce1bed4666f1 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:28.449965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1011/2030] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:28 2022] POST /identity/v3/projects => generated 357 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:28.455749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba35dc-0923-445d-88d6-7b2eaf96b9ba admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:28.469759 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba35dc-0923-445d-88d6-7b2eaf96b9ba admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:28.575752 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241b07a-c326-4937-9785-466ac8c06acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:28.576339 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241b07a-c326-4937-9785-466ac8c06acd admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:28.576635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241b07a-c326-4937-9785-466ac8c06acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:28.576919 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241b07a-c326-4937-9785-466ac8c06acd admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:28.577679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241b07a-c326-4937-9785-466ac8c06acd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:28.580897 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241b07a-c326-4937-9785-466ac8c06acd admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:28.589498 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:28.607924 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1020/2031] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:28 2022] POST /identity/v3/users => generated 279 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:28.612257 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f718e3-b983-4f95-84ab-ce1bed4666f1 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:28.627944 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f718e3-b983-4f95-84ab-ce1bed4666f1 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:28.633504 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e981094-1f7c-441b-b90e-bb256b0a2c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:28.634272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e981094-1f7c-441b-b90e-bb256b0a2c4d admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:28.634777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e981094-1f7c-441b-b90e-bb256b0a2c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:28.635550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e981094-1f7c-441b-b90e-bb256b0a2c4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:28.635997 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e981094-1f7c-441b-b90e-bb256b0a2c4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:28.637770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e981094-1f7c-441b-b90e-bb256b0a2c4d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:28.653184 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1012/2032] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:28.657652 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241b07a-c326-4937-9785-466ac8c06acd admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:28.675740 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241b07a-c326-4937-9785-466ac8c06acd admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:28.681138 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0a13f-78b0-4a6c-8331-fa36fa2726af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:28.682038 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0a13f-78b0-4a6c-8331-fa36fa2726af admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:28.682433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0a13f-78b0-4a6c-8331-fa36fa2726af admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:28.682814 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0a13f-78b0-4a6c-8331-fa36fa2726af admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:28.683669 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0a13f-78b0-4a6c-8331-fa36fa2726af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:28.685423 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0a13f-78b0-4a6c-8331-fa36fa2726af admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:28.713202 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1021/2033] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:28.716239 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e981094-1f7c-441b-b90e-bb256b0a2c4d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:28.728806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e981094-1f7c-441b-b90e-bb256b0a2c4d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:28.737073 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34820360-8a80-4603-a5ae-36e517a74bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:28.738373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34820360-8a80-4603-a5ae-36e517a74bc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:28.738831 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34820360-8a80-4603-a5ae-36e517a74bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:28.739367 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34820360-8a80-4603-a5ae-36e517a74bc5 admin admin] PATH_INFO: `/v3/projects/5073135f33944750848279a08a259581/users/2ac506a6cab34e209d1355c058c23689/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:28.762482 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34820360-8a80-4603-a5ae-36e517a74bc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5073135f33944750848279a08a259581, user_id=2ac506a6cab34e209d1355c058c23689, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:28.765410 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34820360-8a80-4603-a5ae-36e517a74bc5 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:28.780147 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1013/2034] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:28 2022] PUT /identity/v3/projects/5073135f33944750848279a08a259581/users/2ac506a6cab34e209d1355c058c23689/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:28.798069 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de1e25-860b-4b43-a5f1-a75cf26c1354 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:28.798542 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de1e25-860b-4b43-a5f1-a75cf26c1354 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:28.798679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de1e25-860b-4b43-a5f1-a75cf26c1354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:28.837739 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-04de1e25-860b-4b43-a5f1-a75cf26c1354 None None] MFA Rules not processed for user `2ac506a6cab34e209d1355c058c23689`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:28.839435 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-04de1e25-860b-4b43-a5f1-a75cf26c1354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:28.977285 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1022/2035] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:28 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:29.027032 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34820360-8a80-4603-a5ae-36e517a74bc5 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:29.028912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-34820360-8a80-4603-a5ae-36e517a74bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:29.087558 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:29.107448 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34820360-8a80-4603-a5ae-36e517a74bc5 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:29.112648 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0eabd7-b4c0-4b5c-b522-9468563102f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:29.113467 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0eabd7-b4c0-4b5c-b522-9468563102f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:29.114051 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0eabd7-b4c0-4b5c-b522-9468563102f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:29.114449 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0eabd7-b4c0-4b5c-b522-9468563102f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:29.138631 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0eabd7-b4c0-4b5c-b522-9468563102f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:29.140210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0eabd7-b4c0-4b5c-b522-9468563102f6 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:29.152066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1014/2036] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:29 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:29.166881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de1e25-860b-4b43-a5f1-a75cf26c1354 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:29.252990 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de1e25-860b-4b43-a5f1-a75cf26c1354 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:29.260575 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743711f5-2894-4b4a-b91e-f84c670b85b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:29.261492 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743711f5-2894-4b4a-b91e-f84c670b85b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:29.261784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743711f5-2894-4b4a-b91e-f84c670b85b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:29.262066 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743711f5-2894-4b4a-b91e-f84c670b85b2 admin admin] PATH_INFO: `/v3/users/2ac506a6cab34e209d1355c058c23689` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:29.263290 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743711f5-2894-4b4a-b91e-f84c670b85b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ac506a6cab34e209d1355c058c23689)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:29.265269 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743711f5-2894-4b4a-b91e-f84c670b85b2 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:29.319818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-743711f5-2894-4b4a-b91e-f84c670b85b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ac506a6cab34e209d1355c058c23689'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:29.327051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-743711f5-2894-4b4a-b91e-f84c670b85b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ac506a6cab34e209d1355c058c23689'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:29.327761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-743711f5-2894-4b4a-b91e-f84c670b85b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ac506a6cab34e209d1355c058c23689'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:29.337201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-743711f5-2894-4b4a-b91e-f84c670b85b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ac506a6cab34e209d1355c058c23689'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:29.345442 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:29.352118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-743711f5-2894-4b4a-b91e-f84c670b85b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ac506a6cab34e209d1355c058c23689'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:29.359930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1023/2037] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:29 2022] DELETE /identity/v3/users/2ac506a6cab34e209d1355c058c23689 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:29.364688 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c89630-0180-412d-a8bf-8fbd513daa73 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:29.365052 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c89630-0180-412d-a8bf-8fbd513daa73 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:29.365376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c89630-0180-412d-a8bf-8fbd513daa73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:29.383309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-79c89630-0180-412d-a8bf-8fbd513daa73 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:29.384515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-79c89630-0180-412d-a8bf-8fbd513daa73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:29.461846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1015/2038] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:29.462577 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743711f5-2894-4b4a-b91e-f84c670b85b2 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:29.464168 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-743711f5-2894-4b4a-b91e-f84c670b85b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:29.540619 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743711f5-2894-4b4a-b91e-f84c670b85b2 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:29.544379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6983a40-3b63-4e4a-83d6-b702e23baa2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:29.545053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6983a40-3b63-4e4a-83d6-b702e23baa2c service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:29.545380 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6983a40-3b63-4e4a-83d6-b702e23baa2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:29.545670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6983a40-3b63-4e4a-83d6-b702e23baa2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:29.556601 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6983a40-3b63-4e4a-83d6-b702e23baa2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:29.558147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6983a40-3b63-4e4a-83d6-b702e23baa2c service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:29.568654 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1024/2039] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:35:29.982409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c89630-0180-412d-a8bf-8fbd513daa73 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:29.984019 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-79c89630-0180-412d-a8bf-8fbd513daa73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:29.997888 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c89630-0180-412d-a8bf-8fbd513daa73 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:30.015027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e7ce2-c81b-44bb-8171-7984a10d4283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:30.016863 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e7ce2-c81b-44bb-8171-7984a10d4283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:30.017173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e7ce2-c81b-44bb-8171-7984a10d4283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:30.017457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e7ce2-c81b-44bb-8171-7984a10d4283 admin admin] PATH_INFO: `/v3/users/0ebc99db6ab74668bd3d09ec6b89f75d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:30.018914 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e7ce2-c81b-44bb-8171-7984a10d4283 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ebc99db6ab74668bd3d09ec6b89f75d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:30.020876 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e7ce2-c81b-44bb-8171-7984a10d4283 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:30.054791 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6983a40-3b63-4e4a-83d6-b702e23baa2c service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:30.056989 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-d6983a40-3b63-4e4a-83d6-b702e23baa2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:30.075907 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6983a40-3b63-4e4a-83d6-b702e23baa2c service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:30.085283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd9cba-66df-4082-bffb-72d17722e7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:30.086181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd9cba-66df-4082-bffb-72d17722e7f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:30.086503 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd9cba-66df-4082-bffb-72d17722e7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:30.086863 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd9cba-66df-4082-bffb-72d17722e7f8 admin admin] PATH_INFO: `/v3/projects/5073135f33944750848279a08a259581` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:30.088850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd9cba-66df-4082-bffb-72d17722e7f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5073135f33944750848279a08a259581)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:30.090972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd9cba-66df-4082-bffb-72d17722e7f8 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:30.126208 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3b6e7ce2-c81b-44bb-8171-7984a10d4283 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ebc99db6ab74668bd3d09ec6b89f75d'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:30.163077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3b6e7ce2-c81b-44bb-8171-7984a10d4283 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ebc99db6ab74668bd3d09ec6b89f75d'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:30.165441 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3b6e7ce2-c81b-44bb-8171-7984a10d4283 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ebc99db6ab74668bd3d09ec6b89f75d'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:30.185555 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-e7fd9cba-66df-4082-bffb-72d17722e7f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5073135f33944750848279a08a259581', 'request_id': 'req-e7fd9cba-66df-4082-bffb-72d17722e7f8'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:30.193760 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-e7fd9cba-66df-4082-bffb-72d17722e7f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5073135f33944750848279a08a259581', 'request_id': 'req-e7fd9cba-66df-4082-bffb-72d17722e7f8'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:30.197814 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3b6e7ce2-c81b-44bb-8171-7984a10d4283 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ebc99db6ab74668bd3d09ec6b89f75d'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:30.205263 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-e7fd9cba-66df-4082-bffb-72d17722e7f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5073135f33944750848279a08a259581', 'request_id': 'req-e7fd9cba-66df-4082-bffb-72d17722e7f8'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:30.210201 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3b6e7ce2-c81b-44bb-8171-7984a10d4283 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ebc99db6ab74668bd3d09ec6b89f75d'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:30.226740 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1016/2040] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:29 2022] DELETE /identity/v3/users/0ebc99db6ab74668bd3d09ec6b89f75d => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:30.229011 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-e7fd9cba-66df-4082-bffb-72d17722e7f8 admin admin] The token cache is being invalidate because project 5073135f33944750848279a08a259581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:30.229704 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-e7fd9cba-66df-4082-bffb-72d17722e7f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:30.231082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-e7fd9cba-66df-4082-bffb-72d17722e7f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:30.232381 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1d19d-d2ea-4e54-84a9-1e4f7c5b59a5 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:30.232679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1d19d-d2ea-4e54-84a9-1e4f7c5b59a5 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:30.232962 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1d19d-d2ea-4e54-84a9-1e4f7c5b59a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:30.233907 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1025/2041] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:30 2022] DELETE /identity/v3/projects/5073135f33944750848279a08a259581 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:30.244091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1174f31-5d7c-4a8a-86ff-ecdedc61159f None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:30.244607 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1174f31-5d7c-4a8a-86ff-ecdedc61159f None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:30.248115 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1174f31-5d7c-4a8a-86ff-ecdedc61159f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:30.273358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-66d1d19d-d2ea-4e54-84a9-1e4f7c5b59a5 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:30.279107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-66d1d19d-d2ea-4e54-84a9-1e4f7c5b59a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:30.291862 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-d1174f31-5d7c-4a8a-86ff-ecdedc61159f None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:30.301039 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-d1174f31-5d7c-4a8a-86ff-ecdedc61159f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:30.356569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:30.370071 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1017/2042] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:30.381115 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1d19d-d2ea-4e54-84a9-1e4f7c5b59a5 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:30.382265 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1026/2043] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:30.382865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-66d1d19d-d2ea-4e54-84a9-1e4f7c5b59a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:30.389732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af26ab9-b33d-462e-aa39-63ce108b318a None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:30.390019 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af26ab9-b33d-462e-aa39-63ce108b318a None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:30.390310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af26ab9-b33d-462e-aa39-63ce108b318a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:30.411842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-2af26ab9-b33d-462e-aa39-63ce108b318a None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:30.418315 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-2af26ab9-b33d-462e-aa39-63ce108b318a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:30.430554 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1027/2044] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:30.434661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af26ab9-b33d-462e-aa39-63ce108b318a None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:30.444487 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af26ab9-b33d-462e-aa39-63ce108b318a None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:30.452031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda5520-c01f-40f0-b58b-988d8b293ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:30.452733 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda5520-c01f-40f0-b58b-988d8b293ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:30.453039 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda5520-c01f-40f0-b58b-988d8b293ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:30.455905 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda5520-c01f-40f0-b58b-988d8b293ba4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:30.456072 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda5520-c01f-40f0-b58b-988d8b293ba4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:35:30.456345 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda5520-c01f-40f0-b58b-988d8b293ba4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:30.458190 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda5520-c01f-40f0-b58b-988d8b293ba4 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:30.461139 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:30.466732 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1d19d-d2ea-4e54-84a9-1e4f7c5b59a5 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:30.472217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddacf552-fd12-4bf0-82b6-7704d4e9801a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:30.476929 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddacf552-fd12-4bf0-82b6-7704d4e9801a service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:30.477263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddacf552-fd12-4bf0-82b6-7704d4e9801a service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:30.477558 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddacf552-fd12-4bf0-82b6-7704d4e9801a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:30.477930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1028/2045] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:35:30.479644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda5520-c01f-40f0-b58b-988d8b293ba4 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:30.489097 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddacf552-fd12-4bf0-82b6-7704d4e9801a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:30.490577 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddacf552-fd12-4bf0-82b6-7704d4e9801a service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:30.491010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda5520-c01f-40f0-b58b-988d8b293ba4 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:30.499810 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba3df9-b7a3-4830-99d7-4d7ca31c2b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:30.501212 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba3df9-b7a3-4830-99d7-4d7ca31c2b3d admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:30.501503 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba3df9-b7a3-4830-99d7-4d7ca31c2b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:30.501837 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba3df9-b7a3-4830-99d7-4d7ca31c2b3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:30.502574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba3df9-b7a3-4830-99d7-4d7ca31c2b3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:30.502696 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1018/2046] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:30.504501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba3df9-b7a3-4830-99d7-4d7ca31c2b3d admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:30.535894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1029/2047] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:30 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:35:30.537121 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddacf552-fd12-4bf0-82b6-7704d4e9801a service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:30.547209 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddacf552-fd12-4bf0-82b6-7704d4e9801a service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:30.615236 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e5d5e-ddd5-4670-ac04-d8927dce3f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:30.616168 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e5d5e-ddd5-4670-ac04-d8927dce3f51 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:30.616651 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e5d5e-ddd5-4670-ac04-d8927dce3f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:30.617047 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e5d5e-ddd5-4670-ac04-d8927dce3f51 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:30.617905 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e5d5e-ddd5-4670-ac04-d8927dce3f51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:30.620149 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e5d5e-ddd5-4670-ac04-d8927dce3f51 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:30.642191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1019/2048] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:30 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:30.646325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba3df9-b7a3-4830-99d7-4d7ca31c2b3d admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:30.658092 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba3df9-b7a3-4830-99d7-4d7ca31c2b3d admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:30.663247 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f598a-0794-4289-a183-4cd7a321348a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:30.663933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f598a-0794-4289-a183-4cd7a321348a admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:30.664225 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f598a-0794-4289-a183-4cd7a321348a admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:30.664509 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f598a-0794-4289-a183-4cd7a321348a admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:30.665282 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f598a-0794-4289-a183-4cd7a321348a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:30.666906 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f598a-0794-4289-a183-4cd7a321348a admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:30.685179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1030/2049] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:30.686777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e5d5e-ddd5-4670-ac04-d8927dce3f51 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:30.697941 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e5d5e-ddd5-4670-ac04-d8927dce3f51 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:30.702840 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d75eef-0f8d-42ad-9e45-5dd7d4f822af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:30.704213 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d75eef-0f8d-42ad-9e45-5dd7d4f822af admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:30.704524 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d75eef-0f8d-42ad-9e45-5dd7d4f822af admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:30.704822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d75eef-0f8d-42ad-9e45-5dd7d4f822af admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:30.705796 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d75eef-0f8d-42ad-9e45-5dd7d4f822af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:30.707511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d75eef-0f8d-42ad-9e45-5dd7d4f822af admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:30.724124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1020/2050] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:30.725468 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f598a-0794-4289-a183-4cd7a321348a admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:30.736683 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f598a-0794-4289-a183-4cd7a321348a admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:30.741990 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb7694-782b-49ea-9478-642330d1780f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:30.742890 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb7694-782b-49ea-9478-642330d1780f admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:30.743190 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb7694-782b-49ea-9478-642330d1780f admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:30.743477 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb7694-782b-49ea-9478-642330d1780f admin admin] PATH_INFO: `/v3/projects/a7d47af741ca46d4a3c107ea044d6e31/users/b684a837ab01486292c2a3957d4025da/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:30.760469 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb7694-782b-49ea-9478-642330d1780f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7d47af741ca46d4a3c107ea044d6e31, user_id=b684a837ab01486292c2a3957d4025da, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:30.763467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb7694-782b-49ea-9478-642330d1780f admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:30.780603 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1031/2051] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:30 2022] PUT /identity/v3/projects/a7d47af741ca46d4a3c107ea044d6e31/users/b684a837ab01486292c2a3957d4025da/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:30.788619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6672736-985e-4b56-9ce6-71c58c78c7d2 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:30.788920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6672736-985e-4b56-9ce6-71c58c78c7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:30.789238 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6672736-985e-4b56-9ce6-71c58c78c7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:30.811094 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb7694-782b-49ea-9478-642330d1780f admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:30.813584 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-47fb7694-782b-49ea-9478-642330d1780f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:30.838229 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-f6672736-985e-4b56-9ce6-71c58c78c7d2 None None] MFA Rules not processed for user `b684a837ab01486292c2a3957d4025da`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:30.839469 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f6672736-985e-4b56-9ce6-71c58c78c7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:30.915960 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb7694-782b-49ea-9478-642330d1780f admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:30.934146 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ade4e-0be8-484f-91d8-77c23e7e037c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:30.936130 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ade4e-0be8-484f-91d8-77c23e7e037c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:30.936826 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ade4e-0be8-484f-91d8-77c23e7e037c admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:30.937378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ade4e-0be8-484f-91d8-77c23e7e037c admin admin] PATH_INFO: `/v3/projects/72f7d001be1744bfbefb363a5c296497` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:30.941575 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153ade4e-0be8-484f-91d8-77c23e7e037c admin admin] RBAC: Authorizing `identity:delete_project(project_id=72f7d001be1744bfbefb363a5c296497)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:30.944794 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153ade4e-0be8-484f-91d8-77c23e7e037c admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:30.964800 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1021/2052] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:30.970629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6672736-985e-4b56-9ce6-71c58c78c7d2 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:30.987791 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6672736-985e-4b56-9ce6-71c58c78c7d2 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:30.992994 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b810515-2273-45ce-be87-a0612e323aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:30.993700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b810515-2273-45ce-be87-a0612e323aaf admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:30.993991 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b810515-2273-45ce-be87-a0612e323aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:30.994278 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b810515-2273-45ce-be87-a0612e323aaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:30.994991 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b810515-2273-45ce-be87-a0612e323aaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:30.997216 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b810515-2273-45ce-be87-a0612e323aaf admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:31.006816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:31.016842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-153ade4e-0be8-484f-91d8-77c23e7e037c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72f7d001be1744bfbefb363a5c296497', 'request_id': 'req-153ade4e-0be8-484f-91d8-77c23e7e037c'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:31.024162 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-153ade4e-0be8-484f-91d8-77c23e7e037c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72f7d001be1744bfbefb363a5c296497', 'request_id': 'req-153ade4e-0be8-484f-91d8-77c23e7e037c'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:31.031148 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-153ade4e-0be8-484f-91d8-77c23e7e037c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72f7d001be1744bfbefb363a5c296497', 'request_id': 'req-153ade4e-0be8-484f-91d8-77c23e7e037c'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:31.040016 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1022/2053] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:30 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:35:31.041450 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b810515-2273-45ce-be87-a0612e323aaf admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:31.047746 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-153ade4e-0be8-484f-91d8-77c23e7e037c admin admin] The token cache is being invalidate because project 72f7d001be1744bfbefb363a5c296497 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:31.048262 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-153ade4e-0be8-484f-91d8-77c23e7e037c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:31.049300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-153ade4e-0be8-484f-91d8-77c23e7e037c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:31.050941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1032/2054] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:30 2022] DELETE /identity/v3/projects/72f7d001be1744bfbefb363a5c296497 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:35:31.054919 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ade4e-0be8-484f-91d8-77c23e7e037c admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:31.056444 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-153ade4e-0be8-484f-91d8-77c23e7e037c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:31.095544 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:31.126649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b810515-2273-45ce-be87-a0612e323aaf admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:31.133329 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcc233-fe7d-40c7-b17a-9e3dc4136609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:31.134027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcc233-fe7d-40c7-b17a-9e3dc4136609 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:31.134320 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcc233-fe7d-40c7-b17a-9e3dc4136609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:31.134975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ade4e-0be8-484f-91d8-77c23e7e037c admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:31.135093 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcc233-fe7d-40c7-b17a-9e3dc4136609 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:31.135327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fcc233-fe7d-40c7-b17a-9e3dc4136609 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:31.137407 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fcc233-fe7d-40c7-b17a-9e3dc4136609 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:31.139469 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfc882-e998-4f7d-84b6-d23ea67668ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:31.140352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfc882-e998-4f7d-84b6-d23ea67668ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:31.141510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfc882-e998-4f7d-84b6-d23ea67668ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:31.141959 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfc882-e998-4f7d-84b6-d23ea67668ec admin admin] PATH_INFO: `/v3/users/31bc434740004363a869456f7360d567` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:31.147219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfc882-e998-4f7d-84b6-d23ea67668ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=31bc434740004363a869456f7360d567)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:31.150317 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfc882-e998-4f7d-84b6-d23ea67668ec admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:31.166530 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1023/2055] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:31 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:35:31.171062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcc233-fe7d-40c7-b17a-9e3dc4136609 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:31.173809 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-94fcc233-fe7d-40c7-b17a-9e3dc4136609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:31.194257 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcc233-fe7d-40c7-b17a-9e3dc4136609 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:31.199996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256fe58-4eec-42ae-8a6e-ac7a9c334ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:31.200851 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256fe58-4eec-42ae-8a6e-ac7a9c334ade admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:31.201168 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256fe58-4eec-42ae-8a6e-ac7a9c334ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:31.201589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256fe58-4eec-42ae-8a6e-ac7a9c334ade admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:31.202556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256fe58-4eec-42ae-8a6e-ac7a9c334ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:31.205311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256fe58-4eec-42ae-8a6e-ac7a9c334ade admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:31.212735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c5dfc882-e998-4f7d-84b6-d23ea67668ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31bc434740004363a869456f7360d567'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:31.216646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1024/2056] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:31.223554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256fe58-4eec-42ae-8a6e-ac7a9c334ade admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:31.235983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c5dfc882-e998-4f7d-84b6-d23ea67668ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31bc434740004363a869456f7360d567'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:31.235983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c5dfc882-e998-4f7d-84b6-d23ea67668ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31bc434740004363a869456f7360d567'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:31.236282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256fe58-4eec-42ae-8a6e-ac7a9c334ade admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:31.241942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3392e-cd51-4127-a300-878c6e1a43e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:31.242196 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3392e-cd51-4127-a300-878c6e1a43e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:31.242577 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3392e-cd51-4127-a300-878c6e1a43e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:31.242917 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3392e-cd51-4127-a300-878c6e1a43e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:31.243019 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c5dfc882-e998-4f7d-84b6-d23ea67668ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31bc434740004363a869456f7360d567'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:31.244280 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3392e-cd51-4127-a300-878c6e1a43e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:31.245678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3392e-cd51-4127-a300-878c6e1a43e4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:31.258161 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c5dfc882-e998-4f7d-84b6-d23ea67668ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31bc434740004363a869456f7360d567'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:31.259020 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1025/2057] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:31.262965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3392e-cd51-4127-a300-878c6e1a43e4 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:31.264873 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e4c3392e-cd51-4127-a300-878c6e1a43e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:31.268586 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1033/2058] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:31 2022] DELETE /identity/v3/users/31bc434740004363a869456f7360d567 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:35:31.281012 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3392e-cd51-4127-a300-878c6e1a43e4 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:31.370245 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2917c8-e78d-4d4a-aa88-c58960688f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:31.371894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2917c8-e78d-4d4a-aa88-c58960688f7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:31.372181 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2917c8-e78d-4d4a-aa88-c58960688f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:31.372541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2917c8-e78d-4d4a-aa88-c58960688f7a admin admin] PATH_INFO: `/v3/projects/718947af32a94dd886536aed1aadec6a/users/381493ea9129485cb53677e08aeb409d/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:31.396530 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2917c8-e78d-4d4a-aa88-c58960688f7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=718947af32a94dd886536aed1aadec6a, user_id=381493ea9129485cb53677e08aeb409d, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:31.400858 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2917c8-e78d-4d4a-aa88-c58960688f7a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:31.416035 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1026/2059] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:31 2022] PUT /identity/v3/projects/718947af32a94dd886536aed1aadec6a/users/381493ea9129485cb53677e08aeb409d/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:35:31.431070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741876a-7b8d-4de5-8be0-b01f85ab247f None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:31.431521 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741876a-7b8d-4de5-8be0-b01f85ab247f None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:31.431891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741876a-7b8d-4de5-8be0-b01f85ab247f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:31.491180 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-f741876a-7b8d-4de5-8be0-b01f85ab247f None None] MFA Rules not processed for user `381493ea9129485cb53677e08aeb409d`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:31.492684 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-f741876a-7b8d-4de5-8be0-b01f85ab247f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:31.567975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1034/2060] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:31.571914 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2917c8-e78d-4d4a-aa88-c58960688f7a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:31.617575 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:31.641003 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2917c8-e78d-4d4a-aa88-c58960688f7a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:31.646629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca5710-c221-4364-b78a-658e9d76413a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:31.647301 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca5710-c221-4364-b78a-658e9d76413a admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:31.647587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca5710-c221-4364-b78a-658e9d76413a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:31.647866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca5710-c221-4364-b78a-658e9d76413a admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:31.648576 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca5710-c221-4364-b78a-658e9d76413a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:31.650523 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca5710-c221-4364-b78a-658e9d76413a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:31.689459 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1027/2061] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:31 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:35:31.693549 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f741876a-7b8d-4de5-8be0-b01f85ab247f None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:31.707127 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f741876a-7b8d-4de5-8be0-b01f85ab247f None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:31.748074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:31.787375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55d5c9-91bd-459b-81e8-c3b05665d109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:31.788192 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55d5c9-91bd-459b-81e8-c3b05665d109 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:31.788496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55d5c9-91bd-459b-81e8-c3b05665d109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:31.789168 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55d5c9-91bd-459b-81e8-c3b05665d109 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:31.789904 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55d5c9-91bd-459b-81e8-c3b05665d109 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:31.793749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55d5c9-91bd-459b-81e8-c3b05665d109 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:31.835839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1035/2062] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:31 2022] POST /identity/v3/users => generated 298 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:31.839892 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca5710-c221-4364-b78a-658e9d76413a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:31.856486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca5710-c221-4364-b78a-658e9d76413a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:31.864045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ba357-267b-4aed-a96b-24d19048437d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:31.874206 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ba357-267b-4aed-a96b-24d19048437d admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:31.874659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ba357-267b-4aed-a96b-24d19048437d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:31.875066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ba357-267b-4aed-a96b-24d19048437d admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:31.875940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ba357-267b-4aed-a96b-24d19048437d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:31.877796 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ba357-267b-4aed-a96b-24d19048437d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:31.889686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1028/2063] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:31.893781 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55d5c9-91bd-459b-81e8-c3b05665d109 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:31.910792 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55d5c9-91bd-459b-81e8-c3b05665d109 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:31.916800 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d60aa9-0228-41d1-ba65-a79ad95271bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:31.917914 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d60aa9-0228-41d1-ba65-a79ad95271bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:31.918209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d60aa9-0228-41d1-ba65-a79ad95271bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:31.918498 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d60aa9-0228-41d1-ba65-a79ad95271bc admin admin] PATH_INFO: `/v3/projects/99538a8cdff04d71b6773bce14bcb523/users/2b068400329748c48683c54237442e34/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:31.944561 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ba357-267b-4aed-a96b-24d19048437d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:31.946779 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-da6ba357-267b-4aed-a96b-24d19048437d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:31.954048 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d60aa9-0228-41d1-ba65-a79ad95271bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=99538a8cdff04d71b6773bce14bcb523, user_id=2b068400329748c48683c54237442e34, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:31.957174 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d60aa9-0228-41d1-ba65-a79ad95271bc admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:31.971044 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ba357-267b-4aed-a96b-24d19048437d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:31.976213 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1036/2064] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:31 2022] PUT /identity/v3/projects/99538a8cdff04d71b6773bce14bcb523/users/2b068400329748c48683c54237442e34/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:31.980469 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d60aa9-0228-41d1-ba65-a79ad95271bc admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:32.057319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa5f2b-2aa7-415d-abac-1ef6d4adc559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:32.058126 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa5f2b-2aa7-415d-abac-1ef6d4adc559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:32.058420 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa5f2b-2aa7-415d-abac-1ef6d4adc559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:32.058748 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa5f2b-2aa7-415d-abac-1ef6d4adc559 admin admin] PATH_INFO: `/v3/projects/4d5c976ab6d44f8f9906bdcac7a8bfe7` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:32.059980 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa5f2b-2aa7-415d-abac-1ef6d4adc559 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d5c976ab6d44f8f9906bdcac7a8bfe7)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:32.061967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa5f2b-2aa7-415d-abac-1ef6d4adc559 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:32.070620 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d60aa9-0228-41d1-ba65-a79ad95271bc admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:32.075822 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3318b-9e6b-4772-9242-2249ab969dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:32.076441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3318b-9e6b-4772-9242-2249ab969dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:32.076731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3318b-9e6b-4772-9242-2249ab969dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:32.077014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3318b-9e6b-4772-9242-2249ab969dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:32.079198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf3318b-9e6b-4772-9242-2249ab969dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:32.081006 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf3318b-9e6b-4772-9242-2249ab969dc7 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:32.091464 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1037/2065] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:35:32.096950 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3318b-9e6b-4772-9242-2249ab969dc7 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:32.107639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3318b-9e6b-4772-9242-2249ab969dc7 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:32.119106 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-85aa5f2b-2aa7-415d-abac-1ef6d4adc559 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d5c976ab6d44f8f9906bdcac7a8bfe7', 'request_id': 'req-85aa5f2b-2aa7-415d-abac-1ef6d4adc559'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:32.127726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-85aa5f2b-2aa7-415d-abac-1ef6d4adc559 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d5c976ab6d44f8f9906bdcac7a8bfe7', 'request_id': 'req-85aa5f2b-2aa7-415d-abac-1ef6d4adc559'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:32.134489 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-85aa5f2b-2aa7-415d-abac-1ef6d4adc559 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d5c976ab6d44f8f9906bdcac7a8bfe7', 'request_id': 'req-85aa5f2b-2aa7-415d-abac-1ef6d4adc559'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:32.145725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-85aa5f2b-2aa7-415d-abac-1ef6d4adc559 admin admin] The token cache is being invalidate because project 4d5c976ab6d44f8f9906bdcac7a8bfe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:32.146017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-85aa5f2b-2aa7-415d-abac-1ef6d4adc559 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:32.146805 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-85aa5f2b-2aa7-415d-abac-1ef6d4adc559 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:32.148377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1029/2066] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:31 2022] DELETE /identity/v3/projects/4d5c976ab6d44f8f9906bdcac7a8bfe7 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:32.153881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e60f6-52f9-42d5-a3b3-302c413c1f0c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:32.154399 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e60f6-52f9-42d5-a3b3-302c413c1f0c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:32.154839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e60f6-52f9-42d5-a3b3-302c413c1f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:32.171209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84251c22-738c-4bba-ad52-24b375782dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:32.172172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84251c22-738c-4bba-ad52-24b375782dfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:32.172460 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84251c22-738c-4bba-ad52-24b375782dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:32.172745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84251c22-738c-4bba-ad52-24b375782dfa admin admin] PATH_INFO: `/v3/projects/99538a8cdff04d71b6773bce14bcb523/users/2b068400329748c48683c54237442e34/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:32.174806 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84251c22-738c-4bba-ad52-24b375782dfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=99538a8cdff04d71b6773bce14bcb523, user_id=2b068400329748c48683c54237442e34, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:32.175568 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-b36e60f6-52f9-42d5-a3b3-302c413c1f0c None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:32.176803 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b36e60f6-52f9-42d5-a3b3-302c413c1f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:32.177496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84251c22-738c-4bba-ad52-24b375782dfa admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:32.185048 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1030/2067] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:35:32.189324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1038/2068] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:32 2022] PUT /identity/v3/projects/99538a8cdff04d71b6773bce14bcb523/users/2b068400329748c48683c54237442e34/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:32.193698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe2cec-7346-4f47-bc76-d0657f315e05 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:32.194002 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe2cec-7346-4f47-bc76-d0657f315e05 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:32.194337 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe2cec-7346-4f47-bc76-d0657f315e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:32.198918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0086a-b8cc-460f-8667-cb2def3d18b0 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:32.199298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0086a-b8cc-460f-8667-cb2def3d18b0 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:32.199704 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0086a-b8cc-460f-8667-cb2def3d18b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:32.216267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-20fe2cec-7346-4f47-bc76-d0657f315e05 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:32.217516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-20fe2cec-7346-4f47-bc76-d0657f315e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:32.240597 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-b7c0086a-b8cc-460f-8667-cb2def3d18b0 None None] MFA Rules not processed for user `2b068400329748c48683c54237442e34`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:32.241524 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b7c0086a-b8cc-460f-8667-cb2def3d18b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:32.252215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:32.278914 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:32.301333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1031/2069] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:32.304493 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe2cec-7346-4f47-bc76-d0657f315e05 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:32.316475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe2cec-7346-4f47-bc76-d0657f315e05 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:32.321831 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b55698-fb61-4e7d-8c51-cae76d8f6b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:32.322931 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b55698-fb61-4e7d-8c51-cae76d8f6b00 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:32.323365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b55698-fb61-4e7d-8c51-cae76d8f6b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:32.323810 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b55698-fb61-4e7d-8c51-cae76d8f6b00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:32.324779 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b55698-fb61-4e7d-8c51-cae76d8f6b00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:35:32.325095 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b55698-fb61-4e7d-8c51-cae76d8f6b00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:32.326848 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b55698-fb61-4e7d-8c51-cae76d8f6b00 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:32.344875 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1032/2070] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:35:32.346068 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b55698-fb61-4e7d-8c51-cae76d8f6b00 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:32.353299 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1039/2071] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:35:32.360612 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b55698-fb61-4e7d-8c51-cae76d8f6b00 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:32.366026 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0086a-b8cc-460f-8667-cb2def3d18b0 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:32.366682 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a936c20-e6a0-4311-a051-c42a5bafcb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:32.367597 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a936c20-e6a0-4311-a051-c42a5bafcb10 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:32.367889 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a936c20-e6a0-4311-a051-c42a5bafcb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:32.368169 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a936c20-e6a0-4311-a051-c42a5bafcb10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:32.368611 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b7c0086a-b8cc-460f-8667-cb2def3d18b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:32.369000 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a936c20-e6a0-4311-a051-c42a5bafcb10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:32.371907 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a936c20-e6a0-4311-a051-c42a5bafcb10 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:32.405394 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1033/2072] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:32 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:32.409986 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a936c20-e6a0-4311-a051-c42a5bafcb10 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:32.451930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0086a-b8cc-460f-8667-cb2def3d18b0 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:32.529471 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a936c20-e6a0-4311-a051-c42a5bafcb10 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:32.535204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a30168-badc-4027-89a2-3803d735f691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:32.535894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a30168-badc-4027-89a2-3803d735f691 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:32.536812 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a30168-badc-4027-89a2-3803d735f691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:32.536812 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a30168-badc-4027-89a2-3803d735f691 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:32.537234 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a30168-badc-4027-89a2-3803d735f691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:32.539331 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a30168-badc-4027-89a2-3803d735f691 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:32.542893 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6f7ab-94de-405a-9dcc-56696ef3b80c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:32.543632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6f7ab-94de-405a-9dcc-56696ef3b80c service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:32.543990 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6f7ab-94de-405a-9dcc-56696ef3b80c service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:32.544469 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6f7ab-94de-405a-9dcc-56696ef3b80c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:32.546877 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-92a6f7ab-94de-405a-9dcc-56696ef3b80c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:32.566317 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1034/2073] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:32 2022] POST /identity/v3/users => generated 288 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:35:32.570571 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a30168-badc-4027-89a2-3803d735f691 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:32.581832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a30168-badc-4027-89a2-3803d735f691 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:32.587873 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42db31-f126-44d6-94cf-eb4235aa8352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:32.588557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42db31-f126-44d6-94cf-eb4235aa8352 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:32.588845 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42db31-f126-44d6-94cf-eb4235aa8352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:32.589127 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42db31-f126-44d6-94cf-eb4235aa8352 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:32.589919 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42db31-f126-44d6-94cf-eb4235aa8352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:32.591750 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42db31-f126-44d6-94cf-eb4235aa8352 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:32.605198 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1035/2074] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:32.610085 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42db31-f126-44d6-94cf-eb4235aa8352 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:32.626493 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42db31-f126-44d6-94cf-eb4235aa8352 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:32.631317 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be9d953-43f6-428c-8494-bc9215e35e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:32.631993 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9d953-43f6-428c-8494-bc9215e35e71 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:32.632412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9d953-43f6-428c-8494-bc9215e35e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:32.632701 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9d953-43f6-428c-8494-bc9215e35e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:32.633475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be9d953-43f6-428c-8494-bc9215e35e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:32.635125 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be9d953-43f6-428c-8494-bc9215e35e71 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:32.637885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6f7ab-94de-405a-9dcc-56696ef3b80c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:32.639324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6f7ab-94de-405a-9dcc-56696ef3b80c service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:32.649720 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1040/2075] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:32.650249 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1036/2076] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:35:32.650866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6f7ab-94de-405a-9dcc-56696ef3b80c service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:32.661791 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6f7ab-94de-405a-9dcc-56696ef3b80c service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:32.666543 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636da84b-fdc3-4b9e-a651-35463e6ce868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:32.667458 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636da84b-fdc3-4b9e-a651-35463e6ce868 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:32.667749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636da84b-fdc3-4b9e-a651-35463e6ce868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:32.668032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636da84b-fdc3-4b9e-a651-35463e6ce868 admin admin] PATH_INFO: `/v3/projects/444eb54430b64ae2808e12fbda1d4179/users/da921c396c7b4e3fa217f686c6a9ea44/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:32.684517 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636da84b-fdc3-4b9e-a651-35463e6ce868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=444eb54430b64ae2808e12fbda1d4179, user_id=da921c396c7b4e3fa217f686c6a9ea44, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:32.687206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636da84b-fdc3-4b9e-a651-35463e6ce868 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:32.702941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1041/2077] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:32 2022] PUT /identity/v3/projects/444eb54430b64ae2808e12fbda1d4179/users/da921c396c7b4e3fa217f686c6a9ea44/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:32.715203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c0888-8b57-460f-bc25-19a088c96edf None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:32.715510 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c0888-8b57-460f-bc25-19a088c96edf None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:32.715892 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c0888-8b57-460f-bc25-19a088c96edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:32.758020 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-570c0888-8b57-460f-bc25-19a088c96edf None None] MFA Rules not processed for user `da921c396c7b4e3fa217f686c6a9ea44`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:32.759295 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-570c0888-8b57-460f-bc25-19a088c96edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:32.878318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:32.884570 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1037/2078] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:32.897468 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636da84b-fdc3-4b9e-a651-35463e6ce868 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:32.909187 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:33.020178 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636da84b-fdc3-4b9e-a651-35463e6ce868 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:33.027798 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847028e-1f5a-408c-ac98-b6361b7fe0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:33.028781 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847028e-1f5a-408c-ac98-b6361b7fe0ac admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:33.029411 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847028e-1f5a-408c-ac98-b6361b7fe0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:33.029860 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847028e-1f5a-408c-ac98-b6361b7fe0ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:33.032878 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847028e-1f5a-408c-ac98-b6361b7fe0ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:33.042253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847028e-1f5a-408c-ac98-b6361b7fe0ac admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:33.097130 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1042/2079] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:32 2022] POST /identity/v3/projects => generated 373 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:33.098050 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c0888-8b57-460f-bc25-19a088c96edf None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:33.116892 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c0888-8b57-460f-bc25-19a088c96edf None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:33.181753 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020af51e-3bb3-41ca-8a84-da6b6a7491f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:33.182475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020af51e-3bb3-41ca-8a84-da6b6a7491f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:33.182766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020af51e-3bb3-41ca-8a84-da6b6a7491f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:33.183082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020af51e-3bb3-41ca-8a84-da6b6a7491f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:33.183780 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020af51e-3bb3-41ca-8a84-da6b6a7491f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:33.185889 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020af51e-3bb3-41ca-8a84-da6b6a7491f2 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:33.210631 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1038/2080] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:33 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:35:33.215322 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847028e-1f5a-408c-ac98-b6361b7fe0ac admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:33.226578 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847028e-1f5a-408c-ac98-b6361b7fe0ac admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:33.231983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6649d3e-6755-4682-b9f4-fbdd1be42ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:33.232746 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6649d3e-6755-4682-b9f4-fbdd1be42ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:33.233108 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6649d3e-6755-4682-b9f4-fbdd1be42ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:33.233497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6649d3e-6755-4682-b9f4-fbdd1be42ab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:33.234508 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6649d3e-6755-4682-b9f4-fbdd1be42ab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:33.236784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6649d3e-6755-4682-b9f4-fbdd1be42ab7 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:33.252065 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1043/2081] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:33.256370 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020af51e-3bb3-41ca-8a84-da6b6a7491f2 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:33.268492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020af51e-3bb3-41ca-8a84-da6b6a7491f2 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:33.273186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f91fed-602f-4b37-8196-6e464464cec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:33.274176 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f91fed-602f-4b37-8196-6e464464cec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:33.274528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f91fed-602f-4b37-8196-6e464464cec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:33.274889 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f91fed-602f-4b37-8196-6e464464cec5 admin admin] PATH_INFO: `/v3/projects/77eae86c8e184fa6a9ab900686c798c6/users/48c2e6d6c1c04fe69e9ca865aba0c70f/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:33.290260 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f91fed-602f-4b37-8196-6e464464cec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77eae86c8e184fa6a9ab900686c798c6, user_id=48c2e6d6c1c04fe69e9ca865aba0c70f, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:33.292918 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f91fed-602f-4b37-8196-6e464464cec5 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:33.306169 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1039/2082] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:33 2022] PUT /identity/v3/projects/77eae86c8e184fa6a9ab900686c798c6/users/48c2e6d6c1c04fe69e9ca865aba0c70f/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:33.310300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6649d3e-6755-4682-b9f4-fbdd1be42ab7 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:33.321698 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6649d3e-6755-4682-b9f4-fbdd1be42ab7 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:33.382700 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dcd225-b0c7-42be-abac-d5379429bc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:33.383448 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dcd225-b0c7-42be-abac-d5379429bc57 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:33.383739 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dcd225-b0c7-42be-abac-d5379429bc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:33.384026 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dcd225-b0c7-42be-abac-d5379429bc57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:33.384803 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dcd225-b0c7-42be-abac-d5379429bc57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:33.386445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dcd225-b0c7-42be-abac-d5379429bc57 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:33.398547 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1044/2083] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:33.403136 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f91fed-602f-4b37-8196-6e464464cec5 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:33.415068 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f91fed-602f-4b37-8196-6e464464cec5 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:33.420471 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ad767-8b52-4951-9fa1-0681006ef1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:33.421408 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ad767-8b52-4951-9fa1-0681006ef1a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:33.421732 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ad767-8b52-4951-9fa1-0681006ef1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:33.422027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ad767-8b52-4951-9fa1-0681006ef1a0 admin admin] PATH_INFO: `/v3/projects/77eae86c8e184fa6a9ab900686c798c6/users/48c2e6d6c1c04fe69e9ca865aba0c70f/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:33.424078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ad767-8b52-4951-9fa1-0681006ef1a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77eae86c8e184fa6a9ab900686c798c6, user_id=48c2e6d6c1c04fe69e9ca865aba0c70f, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:33.426797 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ad767-8b52-4951-9fa1-0681006ef1a0 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:33.438637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1040/2084] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:33 2022] PUT /identity/v3/projects/77eae86c8e184fa6a9ab900686c798c6/users/48c2e6d6c1c04fe69e9ca865aba0c70f/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:33.448215 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2669ff0-6002-42d0-ba3f-ce2bbcee7ad4 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:33.448510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2669ff0-6002-42d0-ba3f-ce2bbcee7ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:33.448806 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2669ff0-6002-42d0-ba3f-ce2bbcee7ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:33.480919 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-c2669ff0-6002-42d0-ba3f-ce2bbcee7ad4 None None] MFA Rules not processed for user `48c2e6d6c1c04fe69e9ca865aba0c70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:33.482869 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-c2669ff0-6002-42d0-ba3f-ce2bbcee7ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:33.568628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:33.573712 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1045/2085] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:33.584890 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ad767-8b52-4951-9fa1-0681006ef1a0 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:33.588112 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-760ad767-8b52-4951-9fa1-0681006ef1a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:33.668952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ad767-8b52-4951-9fa1-0681006ef1a0 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:33.671410 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5e4fa-f03b-4cb3-8db0-b44bcdd560a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:33.672387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5e4fa-f03b-4cb3-8db0-b44bcdd560a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:33.672766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5e4fa-f03b-4cb3-8db0-b44bcdd560a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:33.673059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5e4fa-f03b-4cb3-8db0-b44bcdd560a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:33.734078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a5e4fa-f03b-4cb3-8db0-b44bcdd560a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:33.735555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a5e4fa-f03b-4cb3-8db0-b44bcdd560a3 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:33.747912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1041/2086] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:34.355733 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2669ff0-6002-42d0-ba3f-ce2bbcee7ad4 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:34.365332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2669ff0-6002-42d0-ba3f-ce2bbcee7ad4 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:34.368858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28dc58a-a792-4a46-b8d8-82de7fa9563e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:34.369538 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28dc58a-a792-4a46-b8d8-82de7fa9563e service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:34.369834 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28dc58a-a792-4a46-b8d8-82de7fa9563e service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:34.370118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28dc58a-a792-4a46-b8d8-82de7fa9563e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:34.372158 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b28dc58a-a792-4a46-b8d8-82de7fa9563e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:34.456989 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28dc58a-a792-4a46-b8d8-82de7fa9563e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:34.458633 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28dc58a-a792-4a46-b8d8-82de7fa9563e service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:34.471703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1046/2087] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:35.053578 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5e4fa-f03b-4cb3-8db0-b44bcdd560a3 service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:35.063123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5e4fa-f03b-4cb3-8db0-b44bcdd560a3 service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:35.067129 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c374c3e-57d6-44cc-8140-3c594eeeffc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:35.067779 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c374c3e-57d6-44cc-8140-3c594eeeffc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:35.068075 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c374c3e-57d6-44cc-8140-3c594eeeffc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:35.068358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c374c3e-57d6-44cc-8140-3c594eeeffc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:35.074078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:35.155503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c374c3e-57d6-44cc-8140-3c594eeeffc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:35.158398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c374c3e-57d6-44cc-8140-3c594eeeffc0 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:35.171759 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1042/2088] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:39.466198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28dc58a-a792-4a46-b8d8-82de7fa9563e service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:39.492062 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28dc58a-a792-4a46-b8d8-82de7fa9563e service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:39.503168 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e28b69-d055-4065-911e-f578a65a825d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:39.503915 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e28b69-d055-4065-911e-f578a65a825d service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:39.504276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e28b69-d055-4065-911e-f578a65a825d service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:39.504634 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e28b69-d055-4065-911e-f578a65a825d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:39.520658 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e28b69-d055-4065-911e-f578a65a825d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:39.521750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e28b69-d055-4065-911e-f578a65a825d service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:39.549018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1047/2089] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:39 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:40.312505 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0c5d7-9cf6-4098-b3ea-714f171e1318 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:40.313327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0c5d7-9cf6-4098-b3ea-714f171e1318 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:40.313327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0c5d7-9cf6-4098-b3ea-714f171e1318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:40.333681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-03a0c5d7-9cf6-4098-b3ea-714f171e1318 None None] MFA Rules not processed for user `305767c6f53e4239b8c2b209914065eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:40.334906 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-03a0c5d7-9cf6-4098-b3ea-714f171e1318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:40.418863 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1043/2090] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:40 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:40.849513 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e28b69-d055-4065-911e-f578a65a825d service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:40.850998 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-13e28b69-d055-4065-911e-f578a65a825d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:40.934664 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e28b69-d055-4065-911e-f578a65a825d service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:40.941763 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12d449-2921-46cb-967f-be9f0534dadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:40.943242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12d449-2921-46cb-967f-be9f0534dadd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:40.943562 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12d449-2921-46cb-967f-be9f0534dadd admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:40.943936 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12d449-2921-46cb-967f-be9f0534dadd admin admin] PATH_INFO: `/v3/users/cbe7720ab8b7414a8c336de368ae1ec4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:40.945676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12d449-2921-46cb-967f-be9f0534dadd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbe7720ab8b7414a8c336de368ae1ec4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:40.947909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12d449-2921-46cb-967f-be9f0534dadd admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:41.011654 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-ef12d449-2921-46cb-967f-be9f0534dadd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbe7720ab8b7414a8c336de368ae1ec4'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:41.021856 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-ef12d449-2921-46cb-967f-be9f0534dadd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbe7720ab8b7414a8c336de368ae1ec4'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:41.022914 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-ef12d449-2921-46cb-967f-be9f0534dadd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbe7720ab8b7414a8c336de368ae1ec4'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:41.035424 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-ef12d449-2921-46cb-967f-be9f0534dadd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbe7720ab8b7414a8c336de368ae1ec4'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:41.048895 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-ef12d449-2921-46cb-967f-be9f0534dadd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbe7720ab8b7414a8c336de368ae1ec4'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:41.055213 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1048/2091] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:40 2022] DELETE /identity/v3/users/cbe7720ab8b7414a8c336de368ae1ec4 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:41.301206 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0c5d7-9cf6-4098-b3ea-714f171e1318 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:41.322332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0c5d7-9cf6-4098-b3ea-714f171e1318 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:41.396561 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f655b6-6c61-4464-b8bf-dcc390894a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:41.397815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f655b6-6c61-4464-b8bf-dcc390894a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:41.398107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f655b6-6c61-4464-b8bf-dcc390894a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:41.398389 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f655b6-6c61-4464-b8bf-dcc390894a4c admin admin] PATH_INFO: `/v3/projects/4537634610c645db8bdb7c704c09ee71` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:41.400915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f655b6-6c61-4464-b8bf-dcc390894a4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4537634610c645db8bdb7c704c09ee71)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:41.407242 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f655b6-6c61-4464-b8bf-dcc390894a4c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:41.466310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:41.467786 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e6f655b6-6c61-4464-b8bf-dcc390894a4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4537634610c645db8bdb7c704c09ee71', 'request_id': 'req-e6f655b6-6c61-4464-b8bf-dcc390894a4c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:41.480469 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e6f655b6-6c61-4464-b8bf-dcc390894a4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4537634610c645db8bdb7c704c09ee71', 'request_id': 'req-e6f655b6-6c61-4464-b8bf-dcc390894a4c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:41.488327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e6f655b6-6c61-4464-b8bf-dcc390894a4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4537634610c645db8bdb7c704c09ee71', 'request_id': 'req-e6f655b6-6c61-4464-b8bf-dcc390894a4c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:41.500363 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e6f655b6-6c61-4464-b8bf-dcc390894a4c admin admin] The token cache is being invalidate because project 4537634610c645db8bdb7c704c09ee71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:41.500543 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e6f655b6-6c61-4464-b8bf-dcc390894a4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:41.501351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e6f655b6-6c61-4464-b8bf-dcc390894a4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:41.502961 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1044/2092] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:41 2022] DELETE /identity/v3/projects/4537634610c645db8bdb7c704c09ee71 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:41.507933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8e074-b257-47db-9dfa-febbbcfcf017 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:41.508263 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8e074-b257-47db-9dfa-febbbcfcf017 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:41.508552 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8e074-b257-47db-9dfa-febbbcfcf017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:41.517492 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:41.531636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-90f8e074-b257-47db-9dfa-febbbcfcf017 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:41.533801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-90f8e074-b257-47db-9dfa-febbbcfcf017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:41.601014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1049/2093] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:41.609025 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bcabe-a712-4439-a1dd-0ed2529ac4b6 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:41.609560 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bcabe-a712-4439-a1dd-0ed2529ac4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:41.609880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bcabe-a712-4439-a1dd-0ed2529ac4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:41.671218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-017bcabe-a712-4439-a1dd-0ed2529ac4b6 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:41.672819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-017bcabe-a712-4439-a1dd-0ed2529ac4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:41.681192 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1045/2094] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:35:41.685330 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8e074-b257-47db-9dfa-febbbcfcf017 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:41.695650 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8e074-b257-47db-9dfa-febbbcfcf017 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:41.700237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5f317-538a-48c8-898a-98b540487d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:41.700898 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5f317-538a-48c8-898a-98b540487d02 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:41.701218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5f317-538a-48c8-898a-98b540487d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:41.701507 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5f317-538a-48c8-898a-98b540487d02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:41.702254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d5f317-538a-48c8-898a-98b540487d02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:35:41.702573 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d5f317-538a-48c8-898a-98b540487d02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:41.704957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d5f317-538a-48c8-898a-98b540487d02 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:41.716564 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1050/2095] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:35:41.720820 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bcabe-a712-4439-a1dd-0ed2529ac4b6 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:41.730519 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bcabe-a712-4439-a1dd-0ed2529ac4b6 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:41.735829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247a20f-d10a-4d05-8f7f-a5be34122070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:41.736710 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247a20f-d10a-4d05-8f7f-a5be34122070 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:41.737003 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247a20f-d10a-4d05-8f7f-a5be34122070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:41.737325 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247a20f-d10a-4d05-8f7f-a5be34122070 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:41.738043 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247a20f-d10a-4d05-8f7f-a5be34122070 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:41.740076 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247a20f-d10a-4d05-8f7f-a5be34122070 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:41.769025 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1046/2096] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:41 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:41.773202 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5f317-538a-48c8-898a-98b540487d02 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:41.783475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5f317-538a-48c8-898a-98b540487d02 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:41.843526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326567f-5fd1-4b61-a2a7-3b04780bf04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:41.844207 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326567f-5fd1-4b61-a2a7-3b04780bf04c admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:41.844524 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326567f-5fd1-4b61-a2a7-3b04780bf04c admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:41.844792 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326567f-5fd1-4b61-a2a7-3b04780bf04c admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:41.845534 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7326567f-5fd1-4b61-a2a7-3b04780bf04c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:41.847480 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7326567f-5fd1-4b61-a2a7-3b04780bf04c admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:41.868599 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1051/2097] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:41 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:41.872967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247a20f-d10a-4d05-8f7f-a5be34122070 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:41.891127 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247a20f-d10a-4d05-8f7f-a5be34122070 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:41.896060 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e25b5-081c-4b55-ac7f-53c4bf377b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:41.896762 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e25b5-081c-4b55-ac7f-53c4bf377b32 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:41.897050 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e25b5-081c-4b55-ac7f-53c4bf377b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:41.897382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e25b5-081c-4b55-ac7f-53c4bf377b32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:41.898359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e25b5-081c-4b55-ac7f-53c4bf377b32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:41.900097 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e25b5-081c-4b55-ac7f-53c4bf377b32 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:41.913188 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1047/2098] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:41.916716 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326567f-5fd1-4b61-a2a7-3b04780bf04c admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:41.928315 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326567f-5fd1-4b61-a2a7-3b04780bf04c admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:41.933917 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ffdfd1-c6d1-4061-be0d-d5da89bfeda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:41.934670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffdfd1-c6d1-4061-be0d-d5da89bfeda1 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:41.934964 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffdfd1-c6d1-4061-be0d-d5da89bfeda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:41.935280 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffdfd1-c6d1-4061-be0d-d5da89bfeda1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:41.936063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ffdfd1-c6d1-4061-be0d-d5da89bfeda1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:41.941240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ffdfd1-c6d1-4061-be0d-d5da89bfeda1 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:41.961198 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1052/2099] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:41.963253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e25b5-081c-4b55-ac7f-53c4bf377b32 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:41.978803 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e25b5-081c-4b55-ac7f-53c4bf377b32 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:41.983743 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9b9ea-21ba-4f1d-9cd2-d5b6f5b67c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:41.985476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9b9ea-21ba-4f1d-9cd2-d5b6f5b67c78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:41.985914 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9b9ea-21ba-4f1d-9cd2-d5b6f5b67c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:41.986206 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9b9ea-21ba-4f1d-9cd2-d5b6f5b67c78 admin admin] PATH_INFO: `/v3/projects/34fb554e0cf14c7ab3a9f538273c289e/users/2fbaf6d894e64f19bf041e7be20a6242/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:42.008249 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9b9ea-21ba-4f1d-9cd2-d5b6f5b67c78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34fb554e0cf14c7ab3a9f538273c289e, user_id=2fbaf6d894e64f19bf041e7be20a6242, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:42.010088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9b9ea-21ba-4f1d-9cd2-d5b6f5b67c78 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:42.041224 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1048/2100] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:41 2022] PUT /identity/v3/projects/34fb554e0cf14c7ab3a9f538273c289e/users/2fbaf6d894e64f19bf041e7be20a6242/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:35:42.046321 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609da67-df15-4099-b83a-1248cfeabcea None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:42.047508 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609da67-df15-4099-b83a-1248cfeabcea None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:42.047911 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609da67-df15-4099-b83a-1248cfeabcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:42.066475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-f609da67-df15-4099-b83a-1248cfeabcea None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:42.067765 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-f609da67-df15-4099-b83a-1248cfeabcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:42.134964 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1053/2101] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:35:42.144906 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9b9ea-21ba-4f1d-9cd2-d5b6f5b67c78 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:42.146798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-77d9b9ea-21ba-4f1d-9cd2-d5b6f5b67c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:42.209632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9b9ea-21ba-4f1d-9cd2-d5b6f5b67c78 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:42.213334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b470b29-1e5c-4394-93ff-6505ee55c35b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:42.213994 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b470b29-1e5c-4394-93ff-6505ee55c35b service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:42.214285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b470b29-1e5c-4394-93ff-6505ee55c35b service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:42.214565 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b470b29-1e5c-4394-93ff-6505ee55c35b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:42.223755 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b470b29-1e5c-4394-93ff-6505ee55c35b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:42.225215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b470b29-1e5c-4394-93ff-6505ee55c35b service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:42.235070 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1049/2102] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:43.881989 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609da67-df15-4099-b83a-1248cfeabcea None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:43.886610 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-f609da67-df15-4099-b83a-1248cfeabcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:43.914427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609da67-df15-4099-b83a-1248cfeabcea None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:43.927693 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9de89-0674-428d-88db-c68a74249c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:43.929122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9de89-0674-428d-88db-c68a74249c07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:43.929812 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9de89-0674-428d-88db-c68a74249c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:43.930255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9de89-0674-428d-88db-c68a74249c07 admin admin] PATH_INFO: `/v3/users/b684a837ab01486292c2a3957d4025da` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:43.931873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9de89-0674-428d-88db-c68a74249c07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b684a837ab01486292c2a3957d4025da)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:43.935386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9de89-0674-428d-88db-c68a74249c07 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:43.955887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:44.061384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-96a9de89-0674-428d-88db-c68a74249c07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b684a837ab01486292c2a3957d4025da'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:44.084419 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-96a9de89-0674-428d-88db-c68a74249c07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b684a837ab01486292c2a3957d4025da'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:44.085837 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-96a9de89-0674-428d-88db-c68a74249c07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b684a837ab01486292c2a3957d4025da'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:44.106074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-96a9de89-0674-428d-88db-c68a74249c07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b684a837ab01486292c2a3957d4025da'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:44.128007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-96a9de89-0674-428d-88db-c68a74249c07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b684a837ab01486292c2a3957d4025da'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:44.147967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1054/2103] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:43 2022] DELETE /identity/v3/users/b684a837ab01486292c2a3957d4025da => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:44.154348 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22eece-a2c2-4d92-a231-51d08706c701 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:44.154920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22eece-a2c2-4d92-a231-51d08706c701 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:44.155094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22eece-a2c2-4d92-a231-51d08706c701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:44.182582 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-cd22eece-a2c2-4d92-a231-51d08706c701 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:44.184479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-cd22eece-a2c2-4d92-a231-51d08706c701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:44.330322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1050/2104] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:44.339510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9de89-0674-428d-88db-c68a74249c07 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:44.340961 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-96a9de89-0674-428d-88db-c68a74249c07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:44.441444 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9de89-0674-428d-88db-c68a74249c07 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:44.445706 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0da35c-b274-4018-b5ca-07c29221db24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:44.446589 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0da35c-b274-4018-b5ca-07c29221db24 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:44.447126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0da35c-b274-4018-b5ca-07c29221db24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:44.447638 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0da35c-b274-4018-b5ca-07c29221db24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:44.459240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0da35c-b274-4018-b5ca-07c29221db24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:44.461132 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0da35c-b274-4018-b5ca-07c29221db24 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:44.473340 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1055/2105] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:45.374541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22eece-a2c2-4d92-a231-51d08706c701 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:45.376232 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-cd22eece-a2c2-4d92-a231-51d08706c701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:45.395529 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22eece-a2c2-4d92-a231-51d08706c701 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:45.400690 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfd16a-1a33-46ca-9f9c-bd9dc03878a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:45.401694 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfd16a-1a33-46ca-9f9c-bd9dc03878a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:45.402147 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfd16a-1a33-46ca-9f9c-bd9dc03878a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:45.402431 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfd16a-1a33-46ca-9f9c-bd9dc03878a3 admin admin] PATH_INFO: `/v3/projects/a7d47af741ca46d4a3c107ea044d6e31` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:45.403648 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfd16a-1a33-46ca-9f9c-bd9dc03878a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7d47af741ca46d4a3c107ea044d6e31)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:45.406359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfd16a-1a33-46ca-9f9c-bd9dc03878a3 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:45.419472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:45.539318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d8bfd16a-1a33-46ca-9f9c-bd9dc03878a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7d47af741ca46d4a3c107ea044d6e31', 'request_id': 'req-d8bfd16a-1a33-46ca-9f9c-bd9dc03878a3'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:45.558723 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d8bfd16a-1a33-46ca-9f9c-bd9dc03878a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7d47af741ca46d4a3c107ea044d6e31', 'request_id': 'req-d8bfd16a-1a33-46ca-9f9c-bd9dc03878a3'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:45.580061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d8bfd16a-1a33-46ca-9f9c-bd9dc03878a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7d47af741ca46d4a3c107ea044d6e31', 'request_id': 'req-d8bfd16a-1a33-46ca-9f9c-bd9dc03878a3'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:45.621196 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d8bfd16a-1a33-46ca-9f9c-bd9dc03878a3 admin admin] The token cache is being invalidate because project a7d47af741ca46d4a3c107ea044d6e31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:45.622353 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d8bfd16a-1a33-46ca-9f9c-bd9dc03878a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:45.625737 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d8bfd16a-1a33-46ca-9f9c-bd9dc03878a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:45.647822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1051/2106] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:45 2022] DELETE /identity/v3/projects/a7d47af741ca46d4a3c107ea044d6e31 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:35:45.652858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0da35c-b274-4018-b5ca-07c29221db24 service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:45.664736 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0da35c-b274-4018-b5ca-07c29221db24 service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:45.761137 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe33809-7ea2-4981-b3f5-c911623ef6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:45.762135 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe33809-7ea2-4981-b3f5-c911623ef6c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:45.762552 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe33809-7ea2-4981-b3f5-c911623ef6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:45.762845 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe33809-7ea2-4981-b3f5-c911623ef6c2 admin admin] PATH_INFO: `/v3/users/381493ea9129485cb53677e08aeb409d` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:45.764227 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe33809-7ea2-4981-b3f5-c911623ef6c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=381493ea9129485cb53677e08aeb409d)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:45.766672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe33809-7ea2-4981-b3f5-c911623ef6c2 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:45.805032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:45.836022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-ebe33809-7ea2-4981-b3f5-c911623ef6c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '381493ea9129485cb53677e08aeb409d'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:45.849724 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-ebe33809-7ea2-4981-b3f5-c911623ef6c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '381493ea9129485cb53677e08aeb409d'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:45.850637 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-ebe33809-7ea2-4981-b3f5-c911623ef6c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '381493ea9129485cb53677e08aeb409d'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:45.863934 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-ebe33809-7ea2-4981-b3f5-c911623ef6c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '381493ea9129485cb53677e08aeb409d'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:45.921905 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-ebe33809-7ea2-4981-b3f5-c911623ef6c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '381493ea9129485cb53677e08aeb409d'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:45.935075 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1056/2107] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:45 2022] DELETE /identity/v3/users/381493ea9129485cb53677e08aeb409d => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:46.714108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfd16a-1a33-46ca-9f9c-bd9dc03878a3 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:46.715695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d8bfd16a-1a33-46ca-9f9c-bd9dc03878a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:46.741977 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfd16a-1a33-46ca-9f9c-bd9dc03878a3 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:46.858326 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617bc7d-9828-46b8-a600-cabf600b1d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:46.860374 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617bc7d-9828-46b8-a600-cabf600b1d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:46.860844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617bc7d-9828-46b8-a600-cabf600b1d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:46.861480 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617bc7d-9828-46b8-a600-cabf600b1d13 admin admin] PATH_INFO: `/v3/projects/718947af32a94dd886536aed1aadec6a` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:46.865443 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617bc7d-9828-46b8-a600-cabf600b1d13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=718947af32a94dd886536aed1aadec6a)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:46.868519 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617bc7d-9828-46b8-a600-cabf600b1d13 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:46.947765 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6617bc7d-9828-46b8-a600-cabf600b1d13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '718947af32a94dd886536aed1aadec6a', 'request_id': 'req-6617bc7d-9828-46b8-a600-cabf600b1d13'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:46.966015 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6617bc7d-9828-46b8-a600-cabf600b1d13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '718947af32a94dd886536aed1aadec6a', 'request_id': 'req-6617bc7d-9828-46b8-a600-cabf600b1d13'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:46.975329 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6617bc7d-9828-46b8-a600-cabf600b1d13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '718947af32a94dd886536aed1aadec6a', 'request_id': 'req-6617bc7d-9828-46b8-a600-cabf600b1d13'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:46.988865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6617bc7d-9828-46b8-a600-cabf600b1d13 admin admin] The token cache is being invalidate because project 718947af32a94dd886536aed1aadec6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:46.989193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6617bc7d-9828-46b8-a600-cabf600b1d13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:46.989883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6617bc7d-9828-46b8-a600-cabf600b1d13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:46.994571 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1052/2108] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:46 2022] DELETE /identity/v3/projects/718947af32a94dd886536aed1aadec6a => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:46.996116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe33809-7ea2-4981-b3f5-c911623ef6c2 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:46.999760 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-ebe33809-7ea2-4981-b3f5-c911623ef6c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:47.013930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe33809-7ea2-4981-b3f5-c911623ef6c2 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:47.137258 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b5edc-ca2f-46bc-9355-24208c32b5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:47.138476 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b5edc-ca2f-46bc-9355-24208c32b5cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:47.139164 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b5edc-ca2f-46bc-9355-24208c32b5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:47.139810 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b5edc-ca2f-46bc-9355-24208c32b5cb admin admin] PATH_INFO: `/v3/users/2b068400329748c48683c54237442e34` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:47.141375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2b5edc-ca2f-46bc-9355-24208c32b5cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b068400329748c48683c54237442e34)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:47.147755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2b5edc-ca2f-46bc-9355-24208c32b5cb admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:47.215424 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e897457-1663-4b38-a033-bb8dbbe712df None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:47.215858 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e897457-1663-4b38-a033-bb8dbbe712df None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:47.216211 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e897457-1663-4b38-a033-bb8dbbe712df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:47.226063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9f2b5edc-ca2f-46bc-9355-24208c32b5cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b068400329748c48683c54237442e34'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:47.237576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9f2b5edc-ca2f-46bc-9355-24208c32b5cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b068400329748c48683c54237442e34'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:47.238187 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9f2b5edc-ca2f-46bc-9355-24208c32b5cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b068400329748c48683c54237442e34'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:47.252054 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9f2b5edc-ca2f-46bc-9355-24208c32b5cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b068400329748c48683c54237442e34'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:47.261557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-3e897457-1663-4b38-a033-bb8dbbe712df None None] MFA Rules not processed for user `2fbaf6d894e64f19bf041e7be20a6242`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:47.263036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3e897457-1663-4b38-a033-bb8dbbe712df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:47.267669 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9f2b5edc-ca2f-46bc-9355-24208c32b5cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b068400329748c48683c54237442e34'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:47.274337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1057/2109] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:46 2022] DELETE /identity/v3/users/2b068400329748c48683c54237442e34 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:47.283809 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:47.354151 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1053/2110] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:47.581099 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b5edc-ca2f-46bc-9355-24208c32b5cb admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:47.582773 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-9f2b5edc-ca2f-46bc-9355-24208c32b5cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:47.620756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:47.643748 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e897457-1663-4b38-a033-bb8dbbe712df None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:47.645389 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3e897457-1663-4b38-a033-bb8dbbe712df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:47.664008 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e897457-1663-4b38-a033-bb8dbbe712df None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:47.701302 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b5edc-ca2f-46bc-9355-24208c32b5cb admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:47.710106 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d264085-3d74-407c-98ce-5f30617fb60a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:47.710839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d264085-3d74-407c-98ce-5f30617fb60a service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:47.711212 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d264085-3d74-407c-98ce-5f30617fb60a service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:47.711500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d264085-3d74-407c-98ce-5f30617fb60a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:47.748929 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb20a04-9a9a-426b-8a21-3f64535a5f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:47.749937 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb20a04-9a9a-426b-8a21-3f64535a5f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:47.750229 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb20a04-9a9a-426b-8a21-3f64535a5f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:47.750517 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb20a04-9a9a-426b-8a21-3f64535a5f4b admin admin] PATH_INFO: `/v3/projects/99538a8cdff04d71b6773bce14bcb523` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:47.760807 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb20a04-9a9a-426b-8a21-3f64535a5f4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=99538a8cdff04d71b6773bce14bcb523)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:47.762830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb20a04-9a9a-426b-8a21-3f64535a5f4b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:47.793937 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d264085-3d74-407c-98ce-5f30617fb60a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:47.796753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d264085-3d74-407c-98ce-5f30617fb60a service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:47.807467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1058/2111] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:47.824186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-dcb20a04-9a9a-426b-8a21-3f64535a5f4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99538a8cdff04d71b6773bce14bcb523', 'request_id': 'req-dcb20a04-9a9a-426b-8a21-3f64535a5f4b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:47.838700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-dcb20a04-9a9a-426b-8a21-3f64535a5f4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99538a8cdff04d71b6773bce14bcb523', 'request_id': 'req-dcb20a04-9a9a-426b-8a21-3f64535a5f4b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:47.848146 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-dcb20a04-9a9a-426b-8a21-3f64535a5f4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99538a8cdff04d71b6773bce14bcb523', 'request_id': 'req-dcb20a04-9a9a-426b-8a21-3f64535a5f4b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:47.865250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-dcb20a04-9a9a-426b-8a21-3f64535a5f4b admin admin] The token cache is being invalidate because project 99538a8cdff04d71b6773bce14bcb523 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:47.865376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-dcb20a04-9a9a-426b-8a21-3f64535a5f4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:47.866091 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-dcb20a04-9a9a-426b-8a21-3f64535a5f4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:47.873583 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1054/2112] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:47 2022] DELETE /identity/v3/projects/99538a8cdff04d71b6773bce14bcb523 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:35:49.501117 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a0246-4cf3-4656-961d-35504295aabe None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:49.501776 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a0246-4cf3-4656-961d-35504295aabe None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:49.501776 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a0246-4cf3-4656-961d-35504295aabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:49.519644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-ff2a0246-4cf3-4656-961d-35504295aabe None None] MFA Rules not processed for user `2fbaf6d894e64f19bf041e7be20a6242`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:49.520838 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-ff2a0246-4cf3-4656-961d-35504295aabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:49.581408 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1059/2113] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:35:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:57.861103 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:59.573477 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=70 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:07.393382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb20a04-9a9a-426b-8a21-3f64535a5f4b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:07.394913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-dcb20a04-9a9a-426b-8a21-3f64535a5f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:07.473779 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb20a04-9a9a-426b-8a21-3f64535a5f4b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:07.478889 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c60d00-d28c-4510-b1aa-1f28aedee46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:07.479981 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c60d00-d28c-4510-b1aa-1f28aedee46b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:07.480269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c60d00-d28c-4510-b1aa-1f28aedee46b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:07.480601 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c60d00-d28c-4510-b1aa-1f28aedee46b admin admin] PATH_INFO: `/v3/users/f401650914a7409e81287b48c1f3ebca` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:07.482505 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c60d00-d28c-4510-b1aa-1f28aedee46b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f401650914a7409e81287b48c1f3ebca)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:07.483825 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c60d00-d28c-4510-b1aa-1f28aedee46b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:07.536472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-37c60d00-d28c-4510-b1aa-1f28aedee46b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f401650914a7409e81287b48c1f3ebca'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:07.551303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-37c60d00-d28c-4510-b1aa-1f28aedee46b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f401650914a7409e81287b48c1f3ebca'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:07.551988 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-37c60d00-d28c-4510-b1aa-1f28aedee46b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f401650914a7409e81287b48c1f3ebca'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:07.559349 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-37c60d00-d28c-4510-b1aa-1f28aedee46b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f401650914a7409e81287b48c1f3ebca'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:07.568969 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-37c60d00-d28c-4510-b1aa-1f28aedee46b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f401650914a7409e81287b48c1f3ebca'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:07.575185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1055/2114] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:36:07 2022] DELETE /identity/v3/users/f401650914a7409e81287b48c1f3ebca => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:07.581894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c60d00-d28c-4510-b1aa-1f28aedee46b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:07.582290 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce7d51-ad52-4e23-a6e2-8ff8e3a14149 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:07.582748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce7d51-ad52-4e23-a6e2-8ff8e3a14149 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:07.582906 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce7d51-ad52-4e23-a6e2-8ff8e3a14149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:07.583280 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-37c60d00-d28c-4510-b1aa-1f28aedee46b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:07.601796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-cbce7d51-ad52-4e23-a6e2-8ff8e3a14149 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:07.603079 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-cbce7d51-ad52-4e23-a6e2-8ff8e3a14149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:07.650357 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c60d00-d28c-4510-b1aa-1f28aedee46b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:07.655609 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d094dc1-7111-4c60-976e-c2ba34011e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:07.656125 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d094dc1-7111-4c60-976e-c2ba34011e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:07.656408 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d094dc1-7111-4c60-976e-c2ba34011e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:07.656686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d094dc1-7111-4c60-976e-c2ba34011e6a admin admin] PATH_INFO: `/v3/users/da921c396c7b4e3fa217f686c6a9ea44` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:07.657921 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d094dc1-7111-4c60-976e-c2ba34011e6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=da921c396c7b4e3fa217f686c6a9ea44)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:07.659965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d094dc1-7111-4c60-976e-c2ba34011e6a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:07.665455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1060/2115] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:07.675867 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce7d51-ad52-4e23-a6e2-8ff8e3a14149 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:07.678031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-cbce7d51-ad52-4e23-a6e2-8ff8e3a14149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:07.682484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:07.711405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8d094dc1-7111-4c60-976e-c2ba34011e6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da921c396c7b4e3fa217f686c6a9ea44'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:07.725363 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8d094dc1-7111-4c60-976e-c2ba34011e6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da921c396c7b4e3fa217f686c6a9ea44'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:07.726095 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8d094dc1-7111-4c60-976e-c2ba34011e6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da921c396c7b4e3fa217f686c6a9ea44'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:07.734625 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8d094dc1-7111-4c60-976e-c2ba34011e6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da921c396c7b4e3fa217f686c6a9ea44'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:07.742941 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8d094dc1-7111-4c60-976e-c2ba34011e6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da921c396c7b4e3fa217f686c6a9ea44'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:07.746271 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce7d51-ad52-4e23-a6e2-8ff8e3a14149 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:07.750583 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae11c0-3e0e-4bc0-a248-5edc71259a80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:07.751190 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae11c0-3e0e-4bc0-a248-5edc71259a80 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:07.751484 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae11c0-3e0e-4bc0-a248-5edc71259a80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:07.751769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae11c0-3e0e-4bc0-a248-5edc71259a80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:07.752279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1056/2116] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:36:07 2022] DELETE /identity/v3/users/da921c396c7b4e3fa217f686c6a9ea44 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:36:07.753735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-e4ae11c0-3e0e-4bc0-a248-5edc71259a80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:07.757127 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30c7af-7e1b-40d0-acc4-49e5d258a9e0 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:07.757548 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30c7af-7e1b-40d0-acc4-49e5d258a9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:07.757783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30c7af-7e1b-40d0-acc4-49e5d258a9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:07.766567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae11c0-3e0e-4bc0-a248-5edc71259a80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:07.768029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae11c0-3e0e-4bc0-a248-5edc71259a80 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:07.779354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-5a30c7af-7e1b-40d0-acc4-49e5d258a9e0 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:07.781228 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5a30c7af-7e1b-40d0-acc4-49e5d258a9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:07.787687 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1061/2117] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:36:07.857197 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1057/2118] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:07.875613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae11c0-3e0e-4bc0-a248-5edc71259a80 service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:07.877139 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-e4ae11c0-3e0e-4bc0-a248-5edc71259a80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:07.889698 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae11c0-3e0e-4bc0-a248-5edc71259a80 service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:07.956549 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8bf5f8-d73a-4a0e-b939-38abc23dddcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:07.957324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bf5f8-d73a-4a0e-b939-38abc23dddcb service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:07.957621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bf5f8-d73a-4a0e-b939-38abc23dddcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:07.957905 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bf5f8-d73a-4a0e-b939-38abc23dddcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:07.968917 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8bf5f8-d73a-4a0e-b939-38abc23dddcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:07.970599 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8bf5f8-d73a-4a0e-b939-38abc23dddcb service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:07.981475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1062/2119] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:08.150736 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30c7af-7e1b-40d0-acc4-49e5d258a9e0 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:08.152279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5a30c7af-7e1b-40d0-acc4-49e5d258a9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:08.164982 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30c7af-7e1b-40d0-acc4-49e5d258a9e0 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:08.169687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97057be9-00ac-4f7f-97e5-11bbc3cff481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:08.170571 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97057be9-00ac-4f7f-97e5-11bbc3cff481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:08.170975 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97057be9-00ac-4f7f-97e5-11bbc3cff481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:08.171267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97057be9-00ac-4f7f-97e5-11bbc3cff481 admin admin] PATH_INFO: `/v3/projects/4e0539eb35ce48819c617525753a27bd` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:08.172573 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97057be9-00ac-4f7f-97e5-11bbc3cff481 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e0539eb35ce48819c617525753a27bd)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:08.174581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97057be9-00ac-4f7f-97e5-11bbc3cff481 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:08.238162 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-97057be9-00ac-4f7f-97e5-11bbc3cff481 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e0539eb35ce48819c617525753a27bd', 'request_id': 'req-97057be9-00ac-4f7f-97e5-11bbc3cff481'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:08.251134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-97057be9-00ac-4f7f-97e5-11bbc3cff481 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e0539eb35ce48819c617525753a27bd', 'request_id': 'req-97057be9-00ac-4f7f-97e5-11bbc3cff481'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:08.258754 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-97057be9-00ac-4f7f-97e5-11bbc3cff481 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e0539eb35ce48819c617525753a27bd', 'request_id': 'req-97057be9-00ac-4f7f-97e5-11bbc3cff481'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:08.270726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-97057be9-00ac-4f7f-97e5-11bbc3cff481 admin admin] The token cache is being invalidate because project 4e0539eb35ce48819c617525753a27bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:36:08.271036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-97057be9-00ac-4f7f-97e5-11bbc3cff481 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:08.271785 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-97057be9-00ac-4f7f-97e5-11bbc3cff481 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:08.273625 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1058/2120] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:36:08 2022] DELETE /identity/v3/projects/4e0539eb35ce48819c617525753a27bd => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:08.278270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36714ff7-5890-4471-8581-5a5268b57699 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:08.278586 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36714ff7-5890-4471-8581-5a5268b57699 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:08.278866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36714ff7-5890-4471-8581-5a5268b57699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:08.297355 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-36714ff7-5890-4471-8581-5a5268b57699 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:08.298578 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-36714ff7-5890-4471-8581-5a5268b57699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:08.335968 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97057be9-00ac-4f7f-97e5-11bbc3cff481 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:08.339312 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-97057be9-00ac-4f7f-97e5-11bbc3cff481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:08.348134 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:08.375563 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1063/2121] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:08.380521 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:08.383408 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d282d85-3249-4755-b134-01215a779ec4 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:08.383648 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d282d85-3249-4755-b134-01215a779ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:08.383944 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d282d85-3249-4755-b134-01215a779ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:08.407569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-7d282d85-3249-4755-b134-01215a779ec4 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:08.408801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-7d282d85-3249-4755-b134-01215a779ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:08.417619 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1064/2122] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:36:08.422205 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d282d85-3249-4755-b134-01215a779ec4 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:08.424023 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97057be9-00ac-4f7f-97e5-11bbc3cff481 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:08.429691 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9159f-44de-4552-a65a-e1bf1c2ee4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:08.430570 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9159f-44de-4552-a65a-e1bf1c2ee4c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:08.430861 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9159f-44de-4552-a65a-e1bf1c2ee4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:08.431145 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9159f-44de-4552-a65a-e1bf1c2ee4c5 admin admin] PATH_INFO: `/v3/projects/444eb54430b64ae2808e12fbda1d4179` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:08.431394 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d282d85-3249-4755-b134-01215a779ec4 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:08.432341 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9159f-44de-4552-a65a-e1bf1c2ee4c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=444eb54430b64ae2808e12fbda1d4179)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:08.434452 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9159f-44de-4552-a65a-e1bf1c2ee4c5 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:08.437272 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb41ac4-cf7f-45d2-9eeb-1d55f429c845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:08.438123 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb41ac4-cf7f-45d2-9eeb-1d55f429c845 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:08.438351 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb41ac4-cf7f-45d2-9eeb-1d55f429c845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:08.438647 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb41ac4-cf7f-45d2-9eeb-1d55f429c845 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:08.439669 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb41ac4-cf7f-45d2-9eeb-1d55f429c845 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:36:08.439991 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb41ac4-cf7f-45d2-9eeb-1d55f429c845 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:08.441727 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb41ac4-cf7f-45d2-9eeb-1d55f429c845 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:08.454256 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1065/2123] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:36:08.458656 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb41ac4-cf7f-45d2-9eeb-1d55f429c845 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:08.468890 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb41ac4-cf7f-45d2-9eeb-1d55f429c845 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:08.474574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7763af-df1e-4f81-b9f1-253161560a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:08.475280 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7763af-df1e-4f81-b9f1-253161560a70 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:08.475559 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7763af-df1e-4f81-b9f1-253161560a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:08.476770 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7763af-df1e-4f81-b9f1-253161560a70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:08.477519 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7763af-df1e-4f81-b9f1-253161560a70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:08.480088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7763af-df1e-4f81-b9f1-253161560a70 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:08.487674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-47d9159f-44de-4552-a65a-e1bf1c2ee4c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '444eb54430b64ae2808e12fbda1d4179', 'request_id': 'req-47d9159f-44de-4552-a65a-e1bf1c2ee4c5'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:08.509504 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-47d9159f-44de-4552-a65a-e1bf1c2ee4c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '444eb54430b64ae2808e12fbda1d4179', 'request_id': 'req-47d9159f-44de-4552-a65a-e1bf1c2ee4c5'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:08.520679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-47d9159f-44de-4552-a65a-e1bf1c2ee4c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '444eb54430b64ae2808e12fbda1d4179', 'request_id': 'req-47d9159f-44de-4552-a65a-e1bf1c2ee4c5'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:08.530256 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1066/2124] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:36:08 2022] POST /identity/v3/projects => generated 365 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:36:08.534755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7763af-df1e-4f81-b9f1-253161560a70 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:08.549268 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-47d9159f-44de-4552-a65a-e1bf1c2ee4c5 admin admin] The token cache is being invalidate because project 444eb54430b64ae2808e12fbda1d4179 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:36:08.550995 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-47d9159f-44de-4552-a65a-e1bf1c2ee4c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:08.550995 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-47d9159f-44de-4552-a65a-e1bf1c2ee4c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:08.554793 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1059/2125] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:36:08 2022] DELETE /identity/v3/projects/444eb54430b64ae2808e12fbda1d4179 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:08.558597 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9159f-44de-4552-a65a-e1bf1c2ee4c5 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:08.560143 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-47d9159f-44de-4552-a65a-e1bf1c2ee4c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:08.648415 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7763af-df1e-4f81-b9f1-253161560a70 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:08.657022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010f2e0-8600-4650-959b-59ed4b05cb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:08.657927 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010f2e0-8600-4650-959b-59ed4b05cb22 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:08.658367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010f2e0-8600-4650-959b-59ed4b05cb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:08.658723 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010f2e0-8600-4650-959b-59ed4b05cb22 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:08.659527 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010f2e0-8600-4650-959b-59ed4b05cb22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:08.661726 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010f2e0-8600-4650-959b-59ed4b05cb22 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:08.664414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9159f-44de-4552-a65a-e1bf1c2ee4c5 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:08.670409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d3003-5221-4740-b24a-7c6aa1ef9d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:08.671739 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d3003-5221-4740-b24a-7c6aa1ef9d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:08.672640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d3003-5221-4740-b24a-7c6aa1ef9d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:08.673120 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d3003-5221-4740-b24a-7c6aa1ef9d47 admin admin] PATH_INFO: `/v3/users/48c2e6d6c1c04fe69e9ca865aba0c70f` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:08.675070 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d3003-5221-4740-b24a-7c6aa1ef9d47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48c2e6d6c1c04fe69e9ca865aba0c70f)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:08.678467 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d3003-5221-4740-b24a-7c6aa1ef9d47 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:08.700682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1067/2126] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:36:08 2022] POST /identity/v3/users => generated 283 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:36:08.702316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010f2e0-8600-4650-959b-59ed4b05cb22 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:08.704328 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-c010f2e0-8600-4650-959b-59ed4b05cb22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:08.720448 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010f2e0-8600-4650-959b-59ed4b05cb22 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:08.726290 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b1a88-83ab-4d58-b910-e1df9709f818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:08.727289 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b1a88-83ab-4d58-b910-e1df9709f818 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:08.727742 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b1a88-83ab-4d58-b910-e1df9709f818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:08.728236 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b1a88-83ab-4d58-b910-e1df9709f818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:08.729181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b1a88-83ab-4d58-b910-e1df9709f818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:08.731284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b1a88-83ab-4d58-b910-e1df9709f818 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:08.745921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1068/2127] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:36:08.751993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b1a88-83ab-4d58-b910-e1df9709f818 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:08.774297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b1a88-83ab-4d58-b910-e1df9709f818 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:08.787780 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff842cc-fca5-45eb-b54d-043ac978abc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:08.788148 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff842cc-fca5-45eb-b54d-043ac978abc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:08.788365 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff842cc-fca5-45eb-b54d-043ac978abc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:08.790484 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff842cc-fca5-45eb-b54d-043ac978abc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:08.791359 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff842cc-fca5-45eb-b54d-043ac978abc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:08.793586 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff842cc-fca5-45eb-b54d-043ac978abc0 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:08.802924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1f7d3003-5221-4740-b24a-7c6aa1ef9d47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48c2e6d6c1c04fe69e9ca865aba0c70f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:08.820520 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1069/2128] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:36:08.821431 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff842cc-fca5-45eb-b54d-043ac978abc0 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:08.822264 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:08.836348 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1f7d3003-5221-4740-b24a-7c6aa1ef9d47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48c2e6d6c1c04fe69e9ca865aba0c70f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:08.846792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1f7d3003-5221-4740-b24a-7c6aa1ef9d47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48c2e6d6c1c04fe69e9ca865aba0c70f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:08.852468 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1f7d3003-5221-4740-b24a-7c6aa1ef9d47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48c2e6d6c1c04fe69e9ca865aba0c70f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:08.855221 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff842cc-fca5-45eb-b54d-043ac978abc0 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:08.864218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0a349-6b66-486a-b309-f1360f72ba77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:08.865378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0a349-6b66-486a-b309-f1360f72ba77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:08.865803 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0a349-6b66-486a-b309-f1360f72ba77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:08.866326 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0a349-6b66-486a-b309-f1360f72ba77 admin admin] PATH_INFO: `/v3/projects/c5c9deabd6444227b04c98d0eb1b3679/users/1793a3b352da4b43acf698aa5bbce3fe/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:08.869335 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1f7d3003-5221-4740-b24a-7c6aa1ef9d47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48c2e6d6c1c04fe69e9ca865aba0c70f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:08.887355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1060/2129] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:36:08 2022] DELETE /identity/v3/users/48c2e6d6c1c04fe69e9ca865aba0c70f => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:08.911792 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0a349-6b66-486a-b309-f1360f72ba77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5c9deabd6444227b04c98d0eb1b3679, user_id=1793a3b352da4b43acf698aa5bbce3fe, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:08.918511 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0a349-6b66-486a-b309-f1360f72ba77 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:08.944300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1070/2130] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:36:08 2022] PUT /identity/v3/projects/c5c9deabd6444227b04c98d0eb1b3679/users/1793a3b352da4b43acf698aa5bbce3fe/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:08.958349 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8169a-f9f1-4871-91aa-ea253a41feea None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:08.959736 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8169a-f9f1-4871-91aa-ea253a41feea None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:08.960281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8169a-f9f1-4871-91aa-ea253a41feea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:09.004534 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-78d8169a-f9f1-4871-91aa-ea253a41feea None None] MFA Rules not processed for user `1793a3b352da4b43acf698aa5bbce3fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:09.006005 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-78d8169a-f9f1-4871-91aa-ea253a41feea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:09.101236 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1061/2131] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:08 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:09.114201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0a349-6b66-486a-b309-f1360f72ba77 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:09.116413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-69c0a349-6b66-486a-b309-f1360f72ba77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:09.201937 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:09.221072 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0a349-6b66-486a-b309-f1360f72ba77 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:09.225412 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c18fa9-a42e-4866-bafe-3f6b6f44fcf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:09.226146 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c18fa9-a42e-4866-bafe-3f6b6f44fcf7 service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:09.226498 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c18fa9-a42e-4866-bafe-3f6b6f44fcf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:09.226862 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c18fa9-a42e-4866-bafe-3f6b6f44fcf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:09.238307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c18fa9-a42e-4866-bafe-3f6b6f44fcf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:09.239937 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c18fa9-a42e-4866-bafe-3f6b6f44fcf7 service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:09.251790 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1071/2132] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:09 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:09.431407 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d8169a-f9f1-4871-91aa-ea253a41feea None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:09.433057 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-78d8169a-f9f1-4871-91aa-ea253a41feea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:09.565097 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d8169a-f9f1-4871-91aa-ea253a41feea None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:09.582063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ab355-3f44-4ef6-bd50-d4c1d59aaac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:09.582897 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ab355-3f44-4ef6-bd50-d4c1d59aaac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:09.583186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ab355-3f44-4ef6-bd50-d4c1d59aaac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:09.583473 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ab355-3f44-4ef6-bd50-d4c1d59aaac4 admin admin] PATH_INFO: `/v3/projects/77eae86c8e184fa6a9ab900686c798c6` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:09.585455 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ab355-3f44-4ef6-bd50-d4c1d59aaac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77eae86c8e184fa6a9ab900686c798c6)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:09.586827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ab355-3f44-4ef6-bd50-d4c1d59aaac4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:09.687813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-581ab355-3f44-4ef6-bd50-d4c1d59aaac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77eae86c8e184fa6a9ab900686c798c6', 'request_id': 'req-581ab355-3f44-4ef6-bd50-d4c1d59aaac4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:09.698732 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-581ab355-3f44-4ef6-bd50-d4c1d59aaac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77eae86c8e184fa6a9ab900686c798c6', 'request_id': 'req-581ab355-3f44-4ef6-bd50-d4c1d59aaac4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:09.709955 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-581ab355-3f44-4ef6-bd50-d4c1d59aaac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77eae86c8e184fa6a9ab900686c798c6', 'request_id': 'req-581ab355-3f44-4ef6-bd50-d4c1d59aaac4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:09.712824 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:09.726342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-581ab355-3f44-4ef6-bd50-d4c1d59aaac4 admin admin] The token cache is being invalidate because project 77eae86c8e184fa6a9ab900686c798c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:36:09.727028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-581ab355-3f44-4ef6-bd50-d4c1d59aaac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:09.728454 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-581ab355-3f44-4ef6-bd50-d4c1d59aaac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:09.740010 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1062/2133] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:36:09 2022] DELETE /identity/v3/projects/77eae86c8e184fa6a9ab900686c798c6 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:09.745316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97f2c1-7435-4865-b190-650334a8aed5 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:09.745868 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97f2c1-7435-4865-b190-650334a8aed5 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:09.746129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97f2c1-7435-4865-b190-650334a8aed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:09.765679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-6c97f2c1-7435-4865-b190-650334a8aed5 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:09.767635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-6c97f2c1-7435-4865-b190-650334a8aed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:09.842519 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1072/2134] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:09.850851 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2e786-30a5-430b-9339-34b417635f05 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:09.851221 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2e786-30a5-430b-9339-34b417635f05 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:09.851573 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2e786-30a5-430b-9339-34b417635f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:09.871197 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a5a2e786-30a5-430b-9339-34b417635f05 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:09.872491 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a5a2e786-30a5-430b-9339-34b417635f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:09.881044 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1063/2135] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:09.885384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c97f2c1-7435-4865-b190-650334a8aed5 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:09.896103 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c97f2c1-7435-4865-b190-650334a8aed5 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:09.901246 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faede41-a305-4d20-90f7-dc8675dbbe38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:09.901920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faede41-a305-4d20-90f7-dc8675dbbe38 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:09.902214 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faede41-a305-4d20-90f7-dc8675dbbe38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:09.902497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faede41-a305-4d20-90f7-dc8675dbbe38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:09.903236 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faede41-a305-4d20-90f7-dc8675dbbe38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:36:09.903595 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faede41-a305-4d20-90f7-dc8675dbbe38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:09.905739 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faede41-a305-4d20-90f7-dc8675dbbe38 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:09.922047 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1073/2136] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:36:09.923298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2e786-30a5-430b-9339-34b417635f05 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:09.933741 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2e786-30a5-430b-9339-34b417635f05 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:09.939157 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae92bac-d424-4583-9a89-97257a84eb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:09.939828 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae92bac-d424-4583-9a89-97257a84eb69 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:09.940117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae92bac-d424-4583-9a89-97257a84eb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:09.940398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae92bac-d424-4583-9a89-97257a84eb69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:09.942286 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae92bac-d424-4583-9a89-97257a84eb69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:09.944187 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae92bac-d424-4583-9a89-97257a84eb69 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:09.978185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1064/2137] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:36:09 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:36:09.982265 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faede41-a305-4d20-90f7-dc8675dbbe38 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:09.993250 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faede41-a305-4d20-90f7-dc8675dbbe38 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:10.057356 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d3b7c-7a2a-4d38-b583-32cf0f1fe3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:10.058165 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d3b7c-7a2a-4d38-b583-32cf0f1fe3be admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:10.058459 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d3b7c-7a2a-4d38-b583-32cf0f1fe3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:10.058748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d3b7c-7a2a-4d38-b583-32cf0f1fe3be admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:10.059465 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d3b7c-7a2a-4d38-b583-32cf0f1fe3be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:10.061952 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d3b7c-7a2a-4d38-b583-32cf0f1fe3be admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:10.083632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1074/2138] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:36:09 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:36:10.087603 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae92bac-d424-4583-9a89-97257a84eb69 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:10.099329 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae92bac-d424-4583-9a89-97257a84eb69 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:10.104127 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c10661-be1d-4c45-af61-8752b9d82321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:10.104874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c10661-be1d-4c45-af61-8752b9d82321 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:10.105271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c10661-be1d-4c45-af61-8752b9d82321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:10.105626 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c10661-be1d-4c45-af61-8752b9d82321 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:10.106437 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c10661-be1d-4c45-af61-8752b9d82321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:10.108884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c10661-be1d-4c45-af61-8752b9d82321 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:10.119797 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1065/2139] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:36:10.124760 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d3b7c-7a2a-4d38-b583-32cf0f1fe3be admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:10.135164 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d3b7c-7a2a-4d38-b583-32cf0f1fe3be admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:10.140453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659f693-6524-4bf1-9bf9-50203419cf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:10.141143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659f693-6524-4bf1-9bf9-50203419cf3b admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:10.141441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659f693-6524-4bf1-9bf9-50203419cf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:10.141749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659f693-6524-4bf1-9bf9-50203419cf3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:10.142459 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659f693-6524-4bf1-9bf9-50203419cf3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:10.144213 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659f693-6524-4bf1-9bf9-50203419cf3b admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:10.159081 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1075/2140] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:36:10.163148 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c10661-be1d-4c45-af61-8752b9d82321 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:10.173298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c10661-be1d-4c45-af61-8752b9d82321 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:10.177962 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61486d9-d307-4772-b6d5-0b5d0f718c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:10.178849 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61486d9-d307-4772-b6d5-0b5d0f718c55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:10.179137 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61486d9-d307-4772-b6d5-0b5d0f718c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:10.179547 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61486d9-d307-4772-b6d5-0b5d0f718c55 admin admin] PATH_INFO: `/v3/projects/495359e002b945f6a92b904e1dc62429/users/215294f66975414ab1245ef26bf735a7/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:10.195191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61486d9-d307-4772-b6d5-0b5d0f718c55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=495359e002b945f6a92b904e1dc62429, user_id=215294f66975414ab1245ef26bf735a7, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:10.197896 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61486d9-d307-4772-b6d5-0b5d0f718c55 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:10.218563 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1066/2141] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:36:10 2022] PUT /identity/v3/projects/495359e002b945f6a92b904e1dc62429/users/215294f66975414ab1245ef26bf735a7/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:10.228725 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b033bb-e901-4b7c-b8b4-f7023db214f7 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:10.229027 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b033bb-e901-4b7c-b8b4-f7023db214f7 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:10.229346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b033bb-e901-4b7c-b8b4-f7023db214f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:10.263043 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-17b033bb-e901-4b7c-b8b4-f7023db214f7 None None] MFA Rules not processed for user `215294f66975414ab1245ef26bf735a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:10.265172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-17b033bb-e901-4b7c-b8b4-f7023db214f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:10.309631 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:10.338769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1076/2142] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:10.344113 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61486d9-d307-4772-b6d5-0b5d0f718c55 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:10.429440 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61486d9-d307-4772-b6d5-0b5d0f718c55 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:10.438010 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535c9be-a4c8-4c8d-97c2-ef42434d7762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:10.438774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535c9be-a4c8-4c8d-97c2-ef42434d7762 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:10.439052 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535c9be-a4c8-4c8d-97c2-ef42434d7762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:10.439336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535c9be-a4c8-4c8d-97c2-ef42434d7762 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:10.440062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535c9be-a4c8-4c8d-97c2-ef42434d7762 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:10.442362 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535c9be-a4c8-4c8d-97c2-ef42434d7762 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:10.473233 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1067/2143] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:36:10 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:36:10.476975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b033bb-e901-4b7c-b8b4-f7023db214f7 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:10.489328 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b033bb-e901-4b7c-b8b4-f7023db214f7 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:10.550123 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc90799-d0c1-46de-b961-79adc664afca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:10.550567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc90799-d0c1-46de-b961-79adc664afca admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:10.550862 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc90799-d0c1-46de-b961-79adc664afca admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:10.551152 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc90799-d0c1-46de-b961-79adc664afca admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:10.551879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc90799-d0c1-46de-b961-79adc664afca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:10.553853 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc90799-d0c1-46de-b961-79adc664afca admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:10.574879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1077/2144] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:36:10 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:36:10.579509 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535c9be-a4c8-4c8d-97c2-ef42434d7762 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:10.589939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535c9be-a4c8-4c8d-97c2-ef42434d7762 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:10.595129 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae0399-407a-486a-b8aa-88c25f8d54d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:10.595800 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae0399-407a-486a-b8aa-88c25f8d54d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:10.596089 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae0399-407a-486a-b8aa-88c25f8d54d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:10.596365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae0399-407a-486a-b8aa-88c25f8d54d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:10.597105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae0399-407a-486a-b8aa-88c25f8d54d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:10.598713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae0399-407a-486a-b8aa-88c25f8d54d4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:10.609510 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1068/2145] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:36:10.614018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc90799-d0c1-46de-b961-79adc664afca admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:10.624634 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc90799-d0c1-46de-b961-79adc664afca admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:10.629401 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90103c-8d3a-42c1-8a29-99c519ac61e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:10.630302 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90103c-8d3a-42c1-8a29-99c519ac61e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:10.630596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90103c-8d3a-42c1-8a29-99c519ac61e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:10.630879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90103c-8d3a-42c1-8a29-99c519ac61e4 admin admin] PATH_INFO: `/v3/projects/a65207b9a60d4203aac0d10a3c82fa46/users/4693bc2524cd49a88e428da66331a8d7/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:10.646919 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90103c-8d3a-42c1-8a29-99c519ac61e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a65207b9a60d4203aac0d10a3c82fa46, user_id=4693bc2524cd49a88e428da66331a8d7, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:10.649634 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90103c-8d3a-42c1-8a29-99c519ac61e4 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:10.663853 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1078/2146] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:36:10 2022] PUT /identity/v3/projects/a65207b9a60d4203aac0d10a3c82fa46/users/4693bc2524cd49a88e428da66331a8d7/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:10.668340 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae0399-407a-486a-b8aa-88c25f8d54d4 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:10.680901 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae0399-407a-486a-b8aa-88c25f8d54d4 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:10.766356 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bcc4f-a01c-4511-aa2d-8cb693d52a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:10.766564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bcc4f-a01c-4511-aa2d-8cb693d52a29 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:10.766837 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bcc4f-a01c-4511-aa2d-8cb693d52a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:10.767122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bcc4f-a01c-4511-aa2d-8cb693d52a29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:10.767885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2bcc4f-a01c-4511-aa2d-8cb693d52a29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:10.769561 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2bcc4f-a01c-4511-aa2d-8cb693d52a29 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:10.771870 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:10.792859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1069/2147] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:36:10.798894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90103c-8d3a-42c1-8a29-99c519ac61e4 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:10.810621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90103c-8d3a-42c1-8a29-99c519ac61e4 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:10.815947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e7b74-7bf8-4524-bf2a-e64bfd995615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:10.816849 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e7b74-7bf8-4524-bf2a-e64bfd995615 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:10.817138 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e7b74-7bf8-4524-bf2a-e64bfd995615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:10.817470 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e7b74-7bf8-4524-bf2a-e64bfd995615 admin admin] PATH_INFO: `/v3/projects/a65207b9a60d4203aac0d10a3c82fa46/users/4693bc2524cd49a88e428da66331a8d7/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:10.819646 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e7b74-7bf8-4524-bf2a-e64bfd995615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a65207b9a60d4203aac0d10a3c82fa46, user_id=4693bc2524cd49a88e428da66331a8d7, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:10.822324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e7b74-7bf8-4524-bf2a-e64bfd995615 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:10.838667 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1079/2148] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:36:10 2022] PUT /identity/v3/projects/a65207b9a60d4203aac0d10a3c82fa46/users/4693bc2524cd49a88e428da66331a8d7/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:10.849570 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5b8ad-94b5-4592-a225-a97a2ed76491 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:10.849866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5b8ad-94b5-4592-a225-a97a2ed76491 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:10.850149 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5b8ad-94b5-4592-a225-a97a2ed76491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:10.879198 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-63e5b8ad-94b5-4592-a225-a97a2ed76491 None None] MFA Rules not processed for user `4693bc2524cd49a88e428da66331a8d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:10.880423 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-63e5b8ad-94b5-4592-a225-a97a2ed76491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:10.957489 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1070/2149] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:10.977586 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e7b74-7bf8-4524-bf2a-e64bfd995615 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:10.979439 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-ce0e7b74-7bf8-4524-bf2a-e64bfd995615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:11.076876 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e7b74-7bf8-4524-bf2a-e64bfd995615 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:11.080695 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93302bbf-03ce-49c0-b989-d48a465c8e24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:11.081372 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93302bbf-03ce-49c0-b989-d48a465c8e24 service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:11.081666 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93302bbf-03ce-49c0-b989-d48a465c8e24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:11.081956 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93302bbf-03ce-49c0-b989-d48a465c8e24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:11.091293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93302bbf-03ce-49c0-b989-d48a465c8e24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:11.093474 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93302bbf-03ce-49c0-b989-d48a465c8e24 service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:11.104642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1080/2150] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:11.136741 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5b8ad-94b5-4592-a225-a97a2ed76491 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:11.249007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5b8ad-94b5-4592-a225-a97a2ed76491 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:11.261238 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426aee72-3c4f-4776-9e9d-3a62bec1ac97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:11.263289 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426aee72-3c4f-4776-9e9d-3a62bec1ac97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:11.263918 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426aee72-3c4f-4776-9e9d-3a62bec1ac97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:11.264058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426aee72-3c4f-4776-9e9d-3a62bec1ac97 admin admin] PATH_INFO: `/v3/users/215294f66975414ab1245ef26bf735a7` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:11.265957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426aee72-3c4f-4776-9e9d-3a62bec1ac97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=215294f66975414ab1245ef26bf735a7)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:11.270132 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426aee72-3c4f-4776-9e9d-3a62bec1ac97 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:11.350590 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-426aee72-3c4f-4776-9e9d-3a62bec1ac97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '215294f66975414ab1245ef26bf735a7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:11.378052 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-426aee72-3c4f-4776-9e9d-3a62bec1ac97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '215294f66975414ab1245ef26bf735a7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:11.378979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-426aee72-3c4f-4776-9e9d-3a62bec1ac97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '215294f66975414ab1245ef26bf735a7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:11.386970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-426aee72-3c4f-4776-9e9d-3a62bec1ac97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '215294f66975414ab1245ef26bf735a7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:11.397038 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-426aee72-3c4f-4776-9e9d-3a62bec1ac97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '215294f66975414ab1245ef26bf735a7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:11.404175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1071/2151] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:36:11 2022] DELETE /identity/v3/users/215294f66975414ab1245ef26bf735a7 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:11.408515 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ebdac-ab73-402c-9f26-78c00ac36bda None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:11.408907 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ebdac-ab73-402c-9f26-78c00ac36bda None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:11.409239 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ebdac-ab73-402c-9f26-78c00ac36bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:11.430009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-b86ebdac-ab73-402c-9f26-78c00ac36bda None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:11.431660 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b86ebdac-ab73-402c-9f26-78c00ac36bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:11.476165 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:11.500804 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1081/2152] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:11.511124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426aee72-3c4f-4776-9e9d-3a62bec1ac97 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:11.512658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-426aee72-3c4f-4776-9e9d-3a62bec1ac97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:11.519105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:11.583289 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426aee72-3c4f-4776-9e9d-3a62bec1ac97 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:11.587865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b9e55-0bc8-40b5-a34b-0143b91d7d21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:11.588527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b9e55-0bc8-40b5-a34b-0143b91d7d21 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:11.589470 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b9e55-0bc8-40b5-a34b-0143b91d7d21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:11.589775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b9e55-0bc8-40b5-a34b-0143b91d7d21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:11.600616 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b9e55-0bc8-40b5-a34b-0143b91d7d21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:11.602151 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b9e55-0bc8-40b5-a34b-0143b91d7d21 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:11.613930 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1072/2153] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:11.950681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ebdac-ab73-402c-9f26-78c00ac36bda None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:11.955969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b86ebdac-ab73-402c-9f26-78c00ac36bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:11.966237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ebdac-ab73-402c-9f26-78c00ac36bda None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:11.971838 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706ea48-69c2-4aff-b1ce-33f188d72f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:11.972969 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706ea48-69c2-4aff-b1ce-33f188d72f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:11.973173 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706ea48-69c2-4aff-b1ce-33f188d72f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:11.973621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706ea48-69c2-4aff-b1ce-33f188d72f28 admin admin] PATH_INFO: `/v3/projects/495359e002b945f6a92b904e1dc62429` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:11.975192 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706ea48-69c2-4aff-b1ce-33f188d72f28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=495359e002b945f6a92b904e1dc62429)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:11.977093 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706ea48-69c2-4aff-b1ce-33f188d72f28 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:12.028356 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-6706ea48-69c2-4aff-b1ce-33f188d72f28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '495359e002b945f6a92b904e1dc62429', 'request_id': 'req-6706ea48-69c2-4aff-b1ce-33f188d72f28'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:12.034923 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-6706ea48-69c2-4aff-b1ce-33f188d72f28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '495359e002b945f6a92b904e1dc62429', 'request_id': 'req-6706ea48-69c2-4aff-b1ce-33f188d72f28'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:12.041229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-6706ea48-69c2-4aff-b1ce-33f188d72f28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '495359e002b945f6a92b904e1dc62429', 'request_id': 'req-6706ea48-69c2-4aff-b1ce-33f188d72f28'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:12.054778 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-6706ea48-69c2-4aff-b1ce-33f188d72f28 admin admin] The token cache is being invalidate because project 495359e002b945f6a92b904e1dc62429 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:36:12.055074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-6706ea48-69c2-4aff-b1ce-33f188d72f28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:12.055695 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-6706ea48-69c2-4aff-b1ce-33f188d72f28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:12.057281 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1082/2154] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:36:11 2022] DELETE /identity/v3/projects/495359e002b945f6a92b904e1dc62429 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:36:12.062535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b9e55-0bc8-40b5-a34b-0143b91d7d21 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:12.072107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b9e55-0bc8-40b5-a34b-0143b91d7d21 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:12.131650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c3f89-4997-47c3-8f9f-a7e1687fbe54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:12.132451 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c3f89-4997-47c3-8f9f-a7e1687fbe54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:12.132740 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c3f89-4997-47c3-8f9f-a7e1687fbe54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:12.133024 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c3f89-4997-47c3-8f9f-a7e1687fbe54 admin admin] PATH_INFO: `/v3/users/4693bc2524cd49a88e428da66331a8d7` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:12.134262 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c3f89-4997-47c3-8f9f-a7e1687fbe54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4693bc2524cd49a88e428da66331a8d7)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:12.136204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c3f89-4997-47c3-8f9f-a7e1687fbe54 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:12.182494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cd1c3f89-4997-47c3-8f9f-a7e1687fbe54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4693bc2524cd49a88e428da66331a8d7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:12.195192 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cd1c3f89-4997-47c3-8f9f-a7e1687fbe54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4693bc2524cd49a88e428da66331a8d7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:12.195884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cd1c3f89-4997-47c3-8f9f-a7e1687fbe54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4693bc2524cd49a88e428da66331a8d7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:12.202826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cd1c3f89-4997-47c3-8f9f-a7e1687fbe54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4693bc2524cd49a88e428da66331a8d7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:12.211300 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-cd1c3f89-4997-47c3-8f9f-a7e1687fbe54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4693bc2524cd49a88e428da66331a8d7'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:12.217956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1073/2155] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:36:12 2022] DELETE /identity/v3/users/4693bc2524cd49a88e428da66331a8d7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:12.622112 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706ea48-69c2-4aff-b1ce-33f188d72f28 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:12.623687 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-6706ea48-69c2-4aff-b1ce-33f188d72f28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:12.639751 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706ea48-69c2-4aff-b1ce-33f188d72f28 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:12.698346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2cfce0-8d3e-4b6d-9933-44f41db4875f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:12.699217 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2cfce0-8d3e-4b6d-9933-44f41db4875f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:12.699504 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2cfce0-8d3e-4b6d-9933-44f41db4875f admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:12.699783 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2cfce0-8d3e-4b6d-9933-44f41db4875f admin admin] PATH_INFO: `/v3/projects/a65207b9a60d4203aac0d10a3c82fa46` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:12.701548 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2cfce0-8d3e-4b6d-9933-44f41db4875f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a65207b9a60d4203aac0d10a3c82fa46)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:12.704123 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2cfce0-8d3e-4b6d-9933-44f41db4875f admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:12.752430 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-0d2cfce0-8d3e-4b6d-9933-44f41db4875f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a65207b9a60d4203aac0d10a3c82fa46', 'request_id': 'req-0d2cfce0-8d3e-4b6d-9933-44f41db4875f'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:12.758083 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-0d2cfce0-8d3e-4b6d-9933-44f41db4875f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a65207b9a60d4203aac0d10a3c82fa46', 'request_id': 'req-0d2cfce0-8d3e-4b6d-9933-44f41db4875f'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:12.765012 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-0d2cfce0-8d3e-4b6d-9933-44f41db4875f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a65207b9a60d4203aac0d10a3c82fa46', 'request_id': 'req-0d2cfce0-8d3e-4b6d-9933-44f41db4875f'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:12.768853 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:12.779366 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-0d2cfce0-8d3e-4b6d-9933-44f41db4875f admin admin] The token cache is being invalidate because project a65207b9a60d4203aac0d10a3c82fa46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:36:12.780352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-0d2cfce0-8d3e-4b6d-9933-44f41db4875f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:12.781506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-0d2cfce0-8d3e-4b6d-9933-44f41db4875f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:12.786493 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1083/2156] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:36:12 2022] DELETE /identity/v3/projects/a65207b9a60d4203aac0d10a3c82fa46 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:12.788359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893bc5b-f247-4003-9eaf-2adbdb06b043 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:12.788844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893bc5b-f247-4003-9eaf-2adbdb06b043 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:12.788938 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893bc5b-f247-4003-9eaf-2adbdb06b043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:12.809395 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1893bc5b-f247-4003-9eaf-2adbdb06b043 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:12.810696 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1893bc5b-f247-4003-9eaf-2adbdb06b043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:12.830645 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:12.875703 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1074/2157] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:12.884758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23735adc-2ade-4170-9fca-7fe027a39bb4 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:12.885027 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23735adc-2ade-4170-9fca-7fe027a39bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:12.885409 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23735adc-2ade-4170-9fca-7fe027a39bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:12.906229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-23735adc-2ade-4170-9fca-7fe027a39bb4 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:12.907433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-23735adc-2ade-4170-9fca-7fe027a39bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:12.915614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1084/2158] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:12.919837 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893bc5b-f247-4003-9eaf-2adbdb06b043 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:12.929857 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893bc5b-f247-4003-9eaf-2adbdb06b043 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:12.934491 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d80707-dec2-4962-84c0-62c4dccf2f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:12.935159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d80707-dec2-4962-84c0-62c4dccf2f9c admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:12.935961 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d80707-dec2-4962-84c0-62c4dccf2f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:12.936523 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d80707-dec2-4962-84c0-62c4dccf2f9c admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:12.936973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d80707-dec2-4962-84c0-62c4dccf2f9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:36:12.937348 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d80707-dec2-4962-84c0-62c4dccf2f9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:12.938959 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d80707-dec2-4962-84c0-62c4dccf2f9c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:12.950979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1075/2159] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:36:12.955865 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23735adc-2ade-4170-9fca-7fe027a39bb4 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:12.965169 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23735adc-2ade-4170-9fca-7fe027a39bb4 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:12.969860 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7da97-19d5-4c07-a343-d9399ad80cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:12.970515 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7da97-19d5-4c07-a343-d9399ad80cfa admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:12.970805 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7da97-19d5-4c07-a343-d9399ad80cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:12.971087 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7da97-19d5-4c07-a343-d9399ad80cfa admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:12.971788 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7da97-19d5-4c07-a343-d9399ad80cfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:12.973740 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7da97-19d5-4c07-a343-d9399ad80cfa admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:13.002526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1085/2160] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:36:12 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:36:13.006582 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d80707-dec2-4962-84c0-62c4dccf2f9c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:13.022294 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d80707-dec2-4962-84c0-62c4dccf2f9c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:13.080056 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f342c9-e792-4ebc-b37f-80a81e903c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:13.080731 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f342c9-e792-4ebc-b37f-80a81e903c02 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:13.081016 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f342c9-e792-4ebc-b37f-80a81e903c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:13.081326 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f342c9-e792-4ebc-b37f-80a81e903c02 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:13.082013 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f342c9-e792-4ebc-b37f-80a81e903c02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:13.083956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f342c9-e792-4ebc-b37f-80a81e903c02 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:13.103414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1076/2161] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:36:13 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:36:13.107476 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7da97-19d5-4c07-a343-d9399ad80cfa admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:13.118505 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7da97-19d5-4c07-a343-d9399ad80cfa admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:13.123868 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a34aae-3aff-4730-883c-a169014cdbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:13.124560 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a34aae-3aff-4730-883c-a169014cdbd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:13.124845 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a34aae-3aff-4730-883c-a169014cdbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:13.125124 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a34aae-3aff-4730-883c-a169014cdbd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:13.125893 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a34aae-3aff-4730-883c-a169014cdbd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:13.127504 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a34aae-3aff-4730-883c-a169014cdbd0 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:13.140609 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1086/2162] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:36:13.144766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f342c9-e792-4ebc-b37f-80a81e903c02 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:13.156301 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f342c9-e792-4ebc-b37f-80a81e903c02 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:13.160996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023f78e-ff3c-4d45-b63f-95aab5d595eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:13.161696 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023f78e-ff3c-4d45-b63f-95aab5d595eb admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:13.161983 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023f78e-ff3c-4d45-b63f-95aab5d595eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:13.162264 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023f78e-ff3c-4d45-b63f-95aab5d595eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:13.162995 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023f78e-ff3c-4d45-b63f-95aab5d595eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:13.164585 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023f78e-ff3c-4d45-b63f-95aab5d595eb admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:13.176125 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1077/2163] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:36:13.180396 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a34aae-3aff-4730-883c-a169014cdbd0 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:13.191068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a34aae-3aff-4730-883c-a169014cdbd0 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:13.195659 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f4997-08b7-4e0c-9082-bfb2a43bcc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:13.196806 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f4997-08b7-4e0c-9082-bfb2a43bcc70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:13.197107 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f4997-08b7-4e0c-9082-bfb2a43bcc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:13.197461 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f4997-08b7-4e0c-9082-bfb2a43bcc70 admin admin] PATH_INFO: `/v3/projects/4403b75fce084b16b754aedbae6ebb70/users/ace262b7504d464fb828a510b9807de6/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:13.212521 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f4997-08b7-4e0c-9082-bfb2a43bcc70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4403b75fce084b16b754aedbae6ebb70, user_id=ace262b7504d464fb828a510b9807de6, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:13.215206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f4997-08b7-4e0c-9082-bfb2a43bcc70 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:13.228675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1087/2164] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:36:13 2022] PUT /identity/v3/projects/4403b75fce084b16b754aedbae6ebb70/users/ace262b7504d464fb828a510b9807de6/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:13.239629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30508971-3a32-4620-9c4c-a5b31d1f5016 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:13.239922 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30508971-3a32-4620-9c4c-a5b31d1f5016 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:13.240199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30508971-3a32-4620-9c4c-a5b31d1f5016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:13.268395 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-30508971-3a32-4620-9c4c-a5b31d1f5016 None None] MFA Rules not processed for user `ace262b7504d464fb828a510b9807de6`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:13.269646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-30508971-3a32-4620-9c4c-a5b31d1f5016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:13.340372 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1078/2165] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:13.345437 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f4997-08b7-4e0c-9082-bfb2a43bcc70 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:13.406496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f4997-08b7-4e0c-9082-bfb2a43bcc70 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:13.411143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa053bb-2278-4cea-8aae-eb4ddd7be780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:13.411806 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa053bb-2278-4cea-8aae-eb4ddd7be780 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:13.412093 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa053bb-2278-4cea-8aae-eb4ddd7be780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:13.412375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa053bb-2278-4cea-8aae-eb4ddd7be780 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:13.413067 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa053bb-2278-4cea-8aae-eb4ddd7be780 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:13.414991 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa053bb-2278-4cea-8aae-eb4ddd7be780 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:13.440576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1088/2166] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:36:13 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:36:13.444601 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30508971-3a32-4620-9c4c-a5b31d1f5016 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:13.456390 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30508971-3a32-4620-9c4c-a5b31d1f5016 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:13.473092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:13.512998 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2883a33-4329-4701-bd73-e83a279c521f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:13.514327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2883a33-4329-4701-bd73-e83a279c521f admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:13.514636 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2883a33-4329-4701-bd73-e83a279c521f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:13.514878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2883a33-4329-4701-bd73-e83a279c521f admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:13.515591 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2883a33-4329-4701-bd73-e83a279c521f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:13.517561 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2883a33-4329-4701-bd73-e83a279c521f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:13.537892 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1079/2167] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:36:13 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:36:13.541912 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa053bb-2278-4cea-8aae-eb4ddd7be780 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:13.552652 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa053bb-2278-4cea-8aae-eb4ddd7be780 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:13.557368 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a9b6f-2a32-440f-baff-2620e8884ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:13.558107 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a9b6f-2a32-440f-baff-2620e8884ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:13.558464 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a9b6f-2a32-440f-baff-2620e8884ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:13.558832 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a9b6f-2a32-440f-baff-2620e8884ec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:13.559633 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a9b6f-2a32-440f-baff-2620e8884ec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:13.561580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a9b6f-2a32-440f-baff-2620e8884ec7 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:13.572633 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1089/2168] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:36:13.577729 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2883a33-4329-4701-bd73-e83a279c521f admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:13.587744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2883a33-4329-4701-bd73-e83a279c521f admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:13.593158 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c17e67-0228-45cf-b8cf-06c5de73991a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:13.593946 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c17e67-0228-45cf-b8cf-06c5de73991a admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:13.594231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c17e67-0228-45cf-b8cf-06c5de73991a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:13.594510 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c17e67-0228-45cf-b8cf-06c5de73991a admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:13.595229 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c17e67-0228-45cf-b8cf-06c5de73991a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:13.596807 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c17e67-0228-45cf-b8cf-06c5de73991a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:13.607638 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1080/2169] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:36:13.617115 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a9b6f-2a32-440f-baff-2620e8884ec7 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:13.637756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a9b6f-2a32-440f-baff-2620e8884ec7 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:13.644053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf6fd6-5be3-4f4f-9c8c-a2d9f74b9417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:13.645130 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf6fd6-5be3-4f4f-9c8c-a2d9f74b9417 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:13.645729 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf6fd6-5be3-4f4f-9c8c-a2d9f74b9417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:13.646101 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf6fd6-5be3-4f4f-9c8c-a2d9f74b9417 admin admin] PATH_INFO: `/v3/projects/0d1827c6b96947b7a040d8ec44147118/users/fe992b0f45f04b03b5867aea51afd7e4/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:13.662930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf6fd6-5be3-4f4f-9c8c-a2d9f74b9417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d1827c6b96947b7a040d8ec44147118, user_id=fe992b0f45f04b03b5867aea51afd7e4, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:13.666172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf6fd6-5be3-4f4f-9c8c-a2d9f74b9417 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:13.680966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1090/2170] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:36:13 2022] PUT /identity/v3/projects/0d1827c6b96947b7a040d8ec44147118/users/fe992b0f45f04b03b5867aea51afd7e4/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:13.692504 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c0888-c653-481d-bcfc-fe58e65c05d7 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:13.692801 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c0888-c653-481d-bcfc-fe58e65c05d7 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:13.693087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c0888-c653-481d-bcfc-fe58e65c05d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:13.721848 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-615c0888-c653-481d-bcfc-fe58e65c05d7 None None] MFA Rules not processed for user `fe992b0f45f04b03b5867aea51afd7e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:13.723273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-615c0888-c653-481d-bcfc-fe58e65c05d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:13.792413 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1081/2171] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:13.801674 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf6fd6-5be3-4f4f-9c8c-a2d9f74b9417 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:13.847656 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:13.866786 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf6fd6-5be3-4f4f-9c8c-a2d9f74b9417 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:13.872064 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dce8ef-4b44-488e-ad96-90bd27e4da7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:13.872732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dce8ef-4b44-488e-ad96-90bd27e4da7d admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:13.873046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dce8ef-4b44-488e-ad96-90bd27e4da7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:13.873367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dce8ef-4b44-488e-ad96-90bd27e4da7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:13.874096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dce8ef-4b44-488e-ad96-90bd27e4da7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:13.876005 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dce8ef-4b44-488e-ad96-90bd27e4da7d admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:13.901997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1091/2172] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:36:13 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:36:13.906792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c0888-c653-481d-bcfc-fe58e65c05d7 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:13.917932 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c0888-c653-481d-bcfc-fe58e65c05d7 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:14.000844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950efabf-9a32-4b39-81d9-8739c7f3daaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:14.001577 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950efabf-9a32-4b39-81d9-8739c7f3daaf admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:14.001871 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950efabf-9a32-4b39-81d9-8739c7f3daaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:14.002781 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950efabf-9a32-4b39-81d9-8739c7f3daaf admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:14.003494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950efabf-9a32-4b39-81d9-8739c7f3daaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:14.005610 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950efabf-9a32-4b39-81d9-8739c7f3daaf admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:14.030438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1082/2173] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:36:13 2022] POST /identity/v3/users => generated 294 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:36:14.031848 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dce8ef-4b44-488e-ad96-90bd27e4da7d admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:14.042262 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dce8ef-4b44-488e-ad96-90bd27e4da7d admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:14.049752 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6f763-a922-4709-a7f6-0a6b0fec5d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:14.050414 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6f763-a922-4709-a7f6-0a6b0fec5d70 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:14.050706 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6f763-a922-4709-a7f6-0a6b0fec5d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:14.051228 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6f763-a922-4709-a7f6-0a6b0fec5d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:14.051910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad6f763-a922-4709-a7f6-0a6b0fec5d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:14.053664 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad6f763-a922-4709-a7f6-0a6b0fec5d70 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:14.065313 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1092/2174] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:36:14.069583 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950efabf-9a32-4b39-81d9-8739c7f3daaf admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:14.080694 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950efabf-9a32-4b39-81d9-8739c7f3daaf admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:14.085447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10586c80-d5bf-4ae3-b2fc-75a8a025b091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:14.086343 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10586c80-d5bf-4ae3-b2fc-75a8a025b091 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:14.086633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10586c80-d5bf-4ae3-b2fc-75a8a025b091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:14.086916 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10586c80-d5bf-4ae3-b2fc-75a8a025b091 admin admin] PATH_INFO: `/v3/projects/197935ea849249a68da7c680280a6a16/users/a52eae5d6c1d46df85aa8ce1286169b9/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:14.102348 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10586c80-d5bf-4ae3-b2fc-75a8a025b091 admin admin] RBAC: Authorizing `identity:create_grant(project_id=197935ea849249a68da7c680280a6a16, user_id=a52eae5d6c1d46df85aa8ce1286169b9, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:14.105017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10586c80-d5bf-4ae3-b2fc-75a8a025b091 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:14.107230 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:14.120436 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1083/2175] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:36:14 2022] PUT /identity/v3/projects/197935ea849249a68da7c680280a6a16/users/a52eae5d6c1d46df85aa8ce1286169b9/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:14.126429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6f763-a922-4709-a7f6-0a6b0fec5d70 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:14.136312 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6f763-a922-4709-a7f6-0a6b0fec5d70 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:14.192674 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1d926-adc2-481e-8cd6-d94167b261e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:14.193285 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1d926-adc2-481e-8cd6-d94167b261e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:14.193576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1d926-adc2-481e-8cd6-d94167b261e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:14.193859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1d926-adc2-481e-8cd6-d94167b261e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:14.194586 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1d926-adc2-481e-8cd6-d94167b261e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:14.196223 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1d926-adc2-481e-8cd6-d94167b261e2 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:14.207131 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1093/2176] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:36:14.211564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10586c80-d5bf-4ae3-b2fc-75a8a025b091 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:14.222798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10586c80-d5bf-4ae3-b2fc-75a8a025b091 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:14.227767 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3dbed-ed11-4090-ba50-c26e8c45239a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:14.228650 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3dbed-ed11-4090-ba50-c26e8c45239a admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:14.228930 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3dbed-ed11-4090-ba50-c26e8c45239a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:14.229257 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3dbed-ed11-4090-ba50-c26e8c45239a admin admin] PATH_INFO: `/v3/projects/197935ea849249a68da7c680280a6a16/users/a52eae5d6c1d46df85aa8ce1286169b9/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:14.231285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3dbed-ed11-4090-ba50-c26e8c45239a admin admin] RBAC: Authorizing `identity:create_grant(project_id=197935ea849249a68da7c680280a6a16, user_id=a52eae5d6c1d46df85aa8ce1286169b9, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:14.234144 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3dbed-ed11-4090-ba50-c26e8c45239a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:14.245115 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1084/2177] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:36:14 2022] PUT /identity/v3/projects/197935ea849249a68da7c680280a6a16/users/a52eae5d6c1d46df85aa8ce1286169b9/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:36:14.255821 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fa807-d206-4eb0-a7c4-4b1c02466aa5 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:14.256116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fa807-d206-4eb0-a7c4-4b1c02466aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:14.256403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fa807-d206-4eb0-a7c4-4b1c02466aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:14.285044 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-c51fa807-d206-4eb0-a7c4-4b1c02466aa5 None None] MFA Rules not processed for user `a52eae5d6c1d46df85aa8ce1286169b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:14.286295 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-c51fa807-d206-4eb0-a7c4-4b1c02466aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:14.355083 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1094/2178] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:14.367015 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3dbed-ed11-4090-ba50-c26e8c45239a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:14.368752 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7ac3dbed-ed11-4090-ba50-c26e8c45239a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:14.425948 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3dbed-ed11-4090-ba50-c26e8c45239a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:14.429674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c2e83-a24a-4ad7-914a-12447c26f17f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:14.430331 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c2e83-a24a-4ad7-914a-12447c26f17f service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:14.430621 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c2e83-a24a-4ad7-914a-12447c26f17f service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:14.430903 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c2e83-a24a-4ad7-914a-12447c26f17f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:14.488953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c2e83-a24a-4ad7-914a-12447c26f17f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:14.490360 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c2e83-a24a-4ad7-914a-12447c26f17f service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:14.500037 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1085/2179] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:15.982822 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fa807-d206-4eb0-a7c4-4b1c02466aa5 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:15.994087 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fa807-d206-4eb0-a7c4-4b1c02466aa5 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:15.998053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50512e-574e-4587-84af-41531ee372d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:15.998708 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50512e-574e-4587-84af-41531ee372d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:15.998995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50512e-574e-4587-84af-41531ee372d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:15.999276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50512e-574e-4587-84af-41531ee372d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:16.035763 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:16.068142 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee50512e-574e-4587-84af-41531ee372d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:16.070736 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee50512e-574e-4587-84af-41531ee372d7 service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:16.081557 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1095/2180] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:16.261172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabb139-5de3-4200-b546-04917e0c30bd None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:16.261726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabb139-5de3-4200-b546-04917e0c30bd None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:16.263276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabb139-5de3-4200-b546-04917e0c30bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:16.284775 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-cbabb139-5de3-4200-b546-04917e0c30bd None None] MFA Rules not processed for user `cfc2f31f654e486d8505d80aad17d17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:16.286133 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-cbabb139-5de3-4200-b546-04917e0c30bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:16.367858 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1086/2181] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 17:36:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:16.419080 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50512e-574e-4587-84af-41531ee372d7 service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:16.422038 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-ee50512e-574e-4587-84af-41531ee372d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:16.512507 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50512e-574e-4587-84af-41531ee372d7 service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:16.519200 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc8bbb-29a5-4f5d-838b-bf2f148105f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a37d1201d54afdad9b228ef4f2d787', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:16.520246 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc8bbb-29a5-4f5d-838b-bf2f148105f3 service swift] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:16.520539 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc8bbb-29a5-4f5d-838b-bf2f148105f3 service swift] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:16.520819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc8bbb-29a5-4f5d-838b-bf2f148105f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:16.532071 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc8bbb-29a5-4f5d-838b-bf2f148105f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:16.534107 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc8bbb-29a5-4f5d-838b-bf2f148105f3 service swift] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:16.543704 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1096/2182] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 17:36:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:36:16.769022 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabb139-5de3-4200-b546-04917e0c30bd None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:16.784336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabb139-5de3-4200-b546-04917e0c30bd None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:16.788796 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba96a6a-1f6a-4de8-9335-56d582aa05b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:16.789777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba96a6a-1f6a-4de8-9335-56d582aa05b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:16.790069 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba96a6a-1f6a-4de8-9335-56d582aa05b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:16.790356 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba96a6a-1f6a-4de8-9335-56d582aa05b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:16.803813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba96a6a-1f6a-4de8-9335-56d582aa05b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:16.805353 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba96a6a-1f6a-4de8-9335-56d582aa05b2 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:16.818165 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1087/2183] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:16 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:25.122746 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc8bbb-29a5-4f5d-838b-bf2f148105f3 service swift] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:25.284670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc8bbb-29a5-4f5d-838b-bf2f148105f3 service swift] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:25.293523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc8c81-288a-4666-90f3-2616676ed01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:25.299113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc8c81-288a-4666-90f3-2616676ed01e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:25.299746 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc8c81-288a-4666-90f3-2616676ed01e admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:25.300792 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc8c81-288a-4666-90f3-2616676ed01e admin admin] PATH_INFO: `/v3/users/ace262b7504d464fb828a510b9807de6` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:25.303449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc8c81-288a-4666-90f3-2616676ed01e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ace262b7504d464fb828a510b9807de6)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:25.307436 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc8c81-288a-4666-90f3-2616676ed01e admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:25.403903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-fdcc8c81-288a-4666-90f3-2616676ed01e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ace262b7504d464fb828a510b9807de6'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:25.417726 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-fdcc8c81-288a-4666-90f3-2616676ed01e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ace262b7504d464fb828a510b9807de6'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:25.418554 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-fdcc8c81-288a-4666-90f3-2616676ed01e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ace262b7504d464fb828a510b9807de6'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:25.430153 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-fdcc8c81-288a-4666-90f3-2616676ed01e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ace262b7504d464fb828a510b9807de6'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:25.436727 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:25.457456 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-fdcc8c81-288a-4666-90f3-2616676ed01e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ace262b7504d464fb828a510b9807de6'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:25.468169 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1097/2184] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:36:25 2022] DELETE /identity/v3/users/ace262b7504d464fb828a510b9807de6 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:25.470289 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e9881-85d7-406f-a56f-7d4fb48d8fda None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:25.470826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e9881-85d7-406f-a56f-7d4fb48d8fda None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:25.471102 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e9881-85d7-406f-a56f-7d4fb48d8fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:25.494364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0b0e9881-85d7-406f-a56f-7d4fb48d8fda None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:25.496192 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0b0e9881-85d7-406f-a56f-7d4fb48d8fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:25.521622 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:25.583304 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1088/2185] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:25.594873 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc8c81-288a-4666-90f3-2616676ed01e admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:25.596667 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-fdcc8c81-288a-4666-90f3-2616676ed01e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:25.671040 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc8c81-288a-4666-90f3-2616676ed01e admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:25.675451 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccea1b9-8c84-4990-a0f7-41295e4ae6f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:25.676329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccea1b9-8c84-4990-a0f7-41295e4ae6f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:25.676790 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccea1b9-8c84-4990-a0f7-41295e4ae6f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:25.677261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccea1b9-8c84-4990-a0f7-41295e4ae6f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:25.689542 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccea1b9-8c84-4990-a0f7-41295e4ae6f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:25.691252 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccea1b9-8c84-4990-a0f7-41295e4ae6f1 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:25.702919 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1098/2186] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:26.047625 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e9881-85d7-406f-a56f-7d4fb48d8fda None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:26.049464 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0b0e9881-85d7-406f-a56f-7d4fb48d8fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:26.071540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e9881-85d7-406f-a56f-7d4fb48d8fda None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:26.089694 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645874e-0a87-41d7-8202-2da1ca4c699c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:26.090892 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645874e-0a87-41d7-8202-2da1ca4c699c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:26.091364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645874e-0a87-41d7-8202-2da1ca4c699c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:26.093866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645874e-0a87-41d7-8202-2da1ca4c699c admin admin] PATH_INFO: `/v3/projects/4403b75fce084b16b754aedbae6ebb70` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:26.095817 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645874e-0a87-41d7-8202-2da1ca4c699c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4403b75fce084b16b754aedbae6ebb70)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:26.098155 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645874e-0a87-41d7-8202-2da1ca4c699c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:26.168119 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f645874e-0a87-41d7-8202-2da1ca4c699c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4403b75fce084b16b754aedbae6ebb70', 'request_id': 'req-f645874e-0a87-41d7-8202-2da1ca4c699c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:26.178883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f645874e-0a87-41d7-8202-2da1ca4c699c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4403b75fce084b16b754aedbae6ebb70', 'request_id': 'req-f645874e-0a87-41d7-8202-2da1ca4c699c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:26.186575 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f645874e-0a87-41d7-8202-2da1ca4c699c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4403b75fce084b16b754aedbae6ebb70', 'request_id': 'req-f645874e-0a87-41d7-8202-2da1ca4c699c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:26.200383 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f645874e-0a87-41d7-8202-2da1ca4c699c admin admin] The token cache is being invalidate because project 4403b75fce084b16b754aedbae6ebb70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:36:26.202331 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f645874e-0a87-41d7-8202-2da1ca4c699c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:26.203664 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f645874e-0a87-41d7-8202-2da1ca4c699c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:26.208321 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1089/2187] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:36:26 2022] DELETE /identity/v3/projects/4403b75fce084b16b754aedbae6ebb70 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:26.211146 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccea1b9-8c84-4990-a0f7-41295e4ae6f1 service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:26.224893 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccea1b9-8c84-4990-a0f7-41295e4ae6f1 service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:26.319686 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e700392-5597-4087-b709-e59abe16565d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:26.320654 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e700392-5597-4087-b709-e59abe16565d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:26.320868 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e700392-5597-4087-b709-e59abe16565d admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:26.321173 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e700392-5597-4087-b709-e59abe16565d admin admin] PATH_INFO: `/v3/users/fe992b0f45f04b03b5867aea51afd7e4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:26.322368 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e700392-5597-4087-b709-e59abe16565d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe992b0f45f04b03b5867aea51afd7e4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:26.324496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e700392-5597-4087-b709-e59abe16565d admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:26.399005 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-5e700392-5597-4087-b709-e59abe16565d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe992b0f45f04b03b5867aea51afd7e4'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:26.423862 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-5e700392-5597-4087-b709-e59abe16565d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe992b0f45f04b03b5867aea51afd7e4'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:26.424495 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-5e700392-5597-4087-b709-e59abe16565d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe992b0f45f04b03b5867aea51afd7e4'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:26.434251 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-5e700392-5597-4087-b709-e59abe16565d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe992b0f45f04b03b5867aea51afd7e4'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:26.450576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-5e700392-5597-4087-b709-e59abe16565d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe992b0f45f04b03b5867aea51afd7e4'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:26.460362 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1099/2188] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:36:26 2022] DELETE /identity/v3/users/fe992b0f45f04b03b5867aea51afd7e4 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:26.891733 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645874e-0a87-41d7-8202-2da1ca4c699c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:26.893452 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f645874e-0a87-41d7-8202-2da1ca4c699c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:26.910675 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645874e-0a87-41d7-8202-2da1ca4c699c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:26.996297 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5992e07-033c-482c-b976-628d8db44eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:26.997390 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5992e07-033c-482c-b976-628d8db44eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:26.997901 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5992e07-033c-482c-b976-628d8db44eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:26.998730 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5992e07-033c-482c-b976-628d8db44eed admin admin] PATH_INFO: `/v3/projects/0d1827c6b96947b7a040d8ec44147118` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:27.000054 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5992e07-033c-482c-b976-628d8db44eed admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d1827c6b96947b7a040d8ec44147118)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:27.002382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5992e07-033c-482c-b976-628d8db44eed admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:27.070514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b5992e07-033c-482c-b976-628d8db44eed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d1827c6b96947b7a040d8ec44147118', 'request_id': 'req-b5992e07-033c-482c-b976-628d8db44eed'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:27.074847 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:27.091255 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b5992e07-033c-482c-b976-628d8db44eed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d1827c6b96947b7a040d8ec44147118', 'request_id': 'req-b5992e07-033c-482c-b976-628d8db44eed'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:27.120305 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b5992e07-033c-482c-b976-628d8db44eed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d1827c6b96947b7a040d8ec44147118', 'request_id': 'req-b5992e07-033c-482c-b976-628d8db44eed'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:27.139304 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b5992e07-033c-482c-b976-628d8db44eed admin admin] The token cache is being invalidate because project 0d1827c6b96947b7a040d8ec44147118 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:36:27.149473 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b5992e07-033c-482c-b976-628d8db44eed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:27.151251 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b5992e07-033c-482c-b976-628d8db44eed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:27.153216 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1090/2189] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:36:26 2022] DELETE /identity/v3/projects/0d1827c6b96947b7a040d8ec44147118 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:27.158343 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e700392-5597-4087-b709-e59abe16565d admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:27.159914 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-5e700392-5597-4087-b709-e59abe16565d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:27.181051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e700392-5597-4087-b709-e59abe16565d admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:27.190424 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:27.325085 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145808e0-b04c-406e-b7e6-c7d3b015851f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:27.326146 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145808e0-b04c-406e-b7e6-c7d3b015851f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:27.327292 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145808e0-b04c-406e-b7e6-c7d3b015851f admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:27.327598 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145808e0-b04c-406e-b7e6-c7d3b015851f admin admin] PATH_INFO: `/v3/users/a52eae5d6c1d46df85aa8ce1286169b9` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:27.329080 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145808e0-b04c-406e-b7e6-c7d3b015851f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a52eae5d6c1d46df85aa8ce1286169b9)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:27.331265 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145808e0-b04c-406e-b7e6-c7d3b015851f admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:27.395962 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-145808e0-b04c-406e-b7e6-c7d3b015851f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a52eae5d6c1d46df85aa8ce1286169b9'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:27.403724 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-145808e0-b04c-406e-b7e6-c7d3b015851f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a52eae5d6c1d46df85aa8ce1286169b9'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:27.404487 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-145808e0-b04c-406e-b7e6-c7d3b015851f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a52eae5d6c1d46df85aa8ce1286169b9'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:27.412477 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-145808e0-b04c-406e-b7e6-c7d3b015851f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a52eae5d6c1d46df85aa8ce1286169b9'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:27.438313 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-145808e0-b04c-406e-b7e6-c7d3b015851f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a52eae5d6c1d46df85aa8ce1286169b9'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:27.455742 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1100/2190] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:36:27 2022] DELETE /identity/v3/users/a52eae5d6c1d46df85aa8ce1286169b9 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:36:27.904386 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5992e07-033c-482c-b976-628d8db44eed admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:27.907571 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b5992e07-033c-482c-b976-628d8db44eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:27.929258 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5992e07-033c-482c-b976-628d8db44eed admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:28.055996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059961a-e204-4bab-a642-591aacb41761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:28.057550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059961a-e204-4bab-a642-591aacb41761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:28.058058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059961a-e204-4bab-a642-591aacb41761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:28.058569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059961a-e204-4bab-a642-591aacb41761 admin admin] PATH_INFO: `/v3/projects/197935ea849249a68da7c680280a6a16` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:28.060361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059961a-e204-4bab-a642-591aacb41761 admin admin] RBAC: Authorizing `identity:delete_project(project_id=197935ea849249a68da7c680280a6a16)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:28.063489 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059961a-e204-4bab-a642-591aacb41761 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:28.147290 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3059961a-e204-4bab-a642-591aacb41761 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '197935ea849249a68da7c680280a6a16', 'request_id': 'req-3059961a-e204-4bab-a642-591aacb41761'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:28.166838 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3059961a-e204-4bab-a642-591aacb41761 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '197935ea849249a68da7c680280a6a16', 'request_id': 'req-3059961a-e204-4bab-a642-591aacb41761'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:28.178286 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3059961a-e204-4bab-a642-591aacb41761 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '197935ea849249a68da7c680280a6a16', 'request_id': 'req-3059961a-e204-4bab-a642-591aacb41761'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:28.198194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3059961a-e204-4bab-a642-591aacb41761 admin admin] The token cache is being invalidate because project 197935ea849249a68da7c680280a6a16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:36:28.198690 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3059961a-e204-4bab-a642-591aacb41761 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:28.199585 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3059961a-e204-4bab-a642-591aacb41761 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:28.203071 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1091/2191] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:36:27 2022] DELETE /identity/v3/projects/197935ea849249a68da7c680280a6a16 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:28.210363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38371186-2978-4ecd-9d3d-c5fa5ce818be None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:28.210697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38371186-2978-4ecd-9d3d-c5fa5ce818be None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:28.211108 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38371186-2978-4ecd-9d3d-c5fa5ce818be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:28.239496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-38371186-2978-4ecd-9d3d-c5fa5ce818be None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:28.241695 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-38371186-2978-4ecd-9d3d-c5fa5ce818be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:28.346594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1101/2192] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:28.360528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1cab5e-1bad-4a77-87de-b49d3e0ad1c5 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:28.360912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1cab5e-1bad-4a77-87de-b49d3e0ad1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:28.361297 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1cab5e-1bad-4a77-87de-b49d3e0ad1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:28.381098 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-7d1cab5e-1bad-4a77-87de-b49d3e0ad1c5 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:28.382464 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7d1cab5e-1bad-4a77-87de-b49d3e0ad1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:28.391475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1092/2193] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:28.395877 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38371186-2978-4ecd-9d3d-c5fa5ce818be None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:28.407285 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38371186-2978-4ecd-9d3d-c5fa5ce818be None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:28.412737 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec883a6f-fae3-432a-b0c4-d9f535bae5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:28.413564 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec883a6f-fae3-432a-b0c4-d9f535bae5d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:28.413958 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec883a6f-fae3-432a-b0c4-d9f535bae5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:28.414347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec883a6f-fae3-432a-b0c4-d9f535bae5d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:28.415225 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec883a6f-fae3-432a-b0c4-d9f535bae5d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:36:28.415604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec883a6f-fae3-432a-b0c4-d9f535bae5d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:28.417268 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec883a6f-fae3-432a-b0c4-d9f535bae5d0 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:28.432322 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1102/2194] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:36:28.436345 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1cab5e-1bad-4a77-87de-b49d3e0ad1c5 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:28.447270 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1cab5e-1bad-4a77-87de-b49d3e0ad1c5 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:28.453480 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa42dee-1df8-40e2-b8fd-696e7db982fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:28.454558 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa42dee-1df8-40e2-b8fd-696e7db982fa admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:28.455065 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa42dee-1df8-40e2-b8fd-696e7db982fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:28.455525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa42dee-1df8-40e2-b8fd-696e7db982fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:28.456646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa42dee-1df8-40e2-b8fd-696e7db982fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:28.458923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa42dee-1df8-40e2-b8fd-696e7db982fa admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:28.504758 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1093/2195] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:36:28 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:36:28.506188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec883a6f-fae3-432a-b0c4-d9f535bae5d0 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:28.529425 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec883a6f-fae3-432a-b0c4-d9f535bae5d0 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:28.541950 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:28.648841 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb39cb-c0d6-4992-85fa-9195c1f1f528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:28.650436 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb39cb-c0d6-4992-85fa-9195c1f1f528 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:28.650734 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb39cb-c0d6-4992-85fa-9195c1f1f528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:28.651026 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb39cb-c0d6-4992-85fa-9195c1f1f528 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:28.651756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb39cb-c0d6-4992-85fa-9195c1f1f528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:28.653716 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb39cb-c0d6-4992-85fa-9195c1f1f528 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:28.695056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1103/2196] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:36:28 2022] POST /identity/v3/users => generated 290 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:36:28.699534 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa42dee-1df8-40e2-b8fd-696e7db982fa admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:28.711419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa42dee-1df8-40e2-b8fd-696e7db982fa admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:28.716330 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5df15f-1600-447d-81da-0264a2b55a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:28.719808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5df15f-1600-447d-81da-0264a2b55a08 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:28.719808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5df15f-1600-447d-81da-0264a2b55a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:28.719808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5df15f-1600-447d-81da-0264a2b55a08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:28.719808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5df15f-1600-447d-81da-0264a2b55a08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:28.720772 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5df15f-1600-447d-81da-0264a2b55a08 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:28.734561 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1094/2197] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:36:28.738724 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb39cb-c0d6-4992-85fa-9195c1f1f528 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:28.751790 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb39cb-c0d6-4992-85fa-9195c1f1f528 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:28.757280 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021c501-21eb-4ba7-9260-a514d25b4aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:28.758078 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021c501-21eb-4ba7-9260-a514d25b4aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:28.758482 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021c501-21eb-4ba7-9260-a514d25b4aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:28.759439 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021c501-21eb-4ba7-9260-a514d25b4aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:28.760253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021c501-21eb-4ba7-9260-a514d25b4aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:28.762034 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021c501-21eb-4ba7-9260-a514d25b4aa1 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:28.783739 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1104/2198] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:36:28.786947 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5df15f-1600-447d-81da-0264a2b55a08 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:28.803601 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5df15f-1600-447d-81da-0264a2b55a08 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:28.816933 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3811257-4e3c-402f-a4e2-5a4d7b7469ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:28.818089 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3811257-4e3c-402f-a4e2-5a4d7b7469ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:28.818502 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3811257-4e3c-402f-a4e2-5a4d7b7469ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:28.819280 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3811257-4e3c-402f-a4e2-5a4d7b7469ca admin admin] PATH_INFO: `/v3/projects/314e9434e0ec4c1f9e8d6037143f5ebb/users/faf0a0102da24cedaedd0c67273e275b/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:28.865886 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3811257-4e3c-402f-a4e2-5a4d7b7469ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=314e9434e0ec4c1f9e8d6037143f5ebb, user_id=faf0a0102da24cedaedd0c67273e275b, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:28.869280 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3811257-4e3c-402f-a4e2-5a4d7b7469ca admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:28.910897 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1095/2199] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:36:28 2022] PUT /identity/v3/projects/314e9434e0ec4c1f9e8d6037143f5ebb/users/faf0a0102da24cedaedd0c67273e275b/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:36:28.911657 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed73921-4e9d-47fa-8205-6f0d0903b31e None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:28.911657 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed73921-4e9d-47fa-8205-6f0d0903b31e None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:28.911657 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed73921-4e9d-47fa-8205-6f0d0903b31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:28.960449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-3ed73921-4e9d-47fa-8205-6f0d0903b31e None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:28.961984 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-3ed73921-4e9d-47fa-8205-6f0d0903b31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:29.072635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1105/2200] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:29.079944 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3811257-4e3c-402f-a4e2-5a4d7b7469ca admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:29.081989 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b3811257-4e3c-402f-a4e2-5a4d7b7469ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:29.119429 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:29.283048 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3811257-4e3c-402f-a4e2-5a4d7b7469ca admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:29.290914 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77524ba5-fb6f-40a2-83be-8c92bf0c23ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:29.293488 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77524ba5-fb6f-40a2-83be-8c92bf0c23ff service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:29.294683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77524ba5-fb6f-40a2-83be-8c92bf0c23ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:29.297095 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77524ba5-fb6f-40a2-83be-8c92bf0c23ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:29.330830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77524ba5-fb6f-40a2-83be-8c92bf0c23ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:29.333198 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77524ba5-fb6f-40a2-83be-8c92bf0c23ff service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:29.349309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1096/2201] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:36:33.991775 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8bb7c-86a9-4e83-b0d2-1dcad15d6e58 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:33.992386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8bb7c-86a9-4e83-b0d2-1dcad15d6e58 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:33.992486 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8bb7c-86a9-4e83-b0d2-1dcad15d6e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:34.050245 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-0ce8bb7c-86a9-4e83-b0d2-1dcad15d6e58 None None] MFA Rules not processed for user `209ada6ba12b4bfda10045b4ee9301f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:34.051526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0ce8bb7c-86a9-4e83-b0d2-1dcad15d6e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:34.191897 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1106/2202] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:36.072844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b787fdc-d443-4a08-9086-2c357792fb62 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:36.073903 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b787fdc-d443-4a08-9086-2c357792fb62 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:36.074396 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b787fdc-d443-4a08-9086-2c357792fb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:36.151541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-2b787fdc-d443-4a08-9086-2c357792fb62 None None] MFA Rules not processed for user `faf0a0102da24cedaedd0c67273e275b`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:36.152561 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2b787fdc-d443-4a08-9086-2c357792fb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:36.318782 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1097/2203] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:36.340736 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8bb7c-86a9-4e83-b0d2-1dcad15d6e58 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:36.428330 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:36.443056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8bb7c-86a9-4e83-b0d2-1dcad15d6e58 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:36.451314 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ed627-ec65-4512-9416-23dec23280aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:36.452172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ed627-ec65-4512-9416-23dec23280aa admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:36.452867 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ed627-ec65-4512-9416-23dec23280aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:36.452867 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ed627-ec65-4512-9416-23dec23280aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:36.453923 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ed627-ec65-4512-9416-23dec23280aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:36.456135 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ed627-ec65-4512-9416-23dec23280aa admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:36.542087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b787fdc-d443-4a08-9086-2c357792fb62 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:36.554771 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1107/2204] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:36:36 2022] POST /identity/v3/projects => generated 375 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:36:36.561900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b787fdc-d443-4a08-9086-2c357792fb62 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:36.680057 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab67233-19d9-4468-94b9-c9e937a08aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:36.680783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab67233-19d9-4468-94b9-c9e937a08aae admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:36.681067 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab67233-19d9-4468-94b9-c9e937a08aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:36.681379 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab67233-19d9-4468-94b9-c9e937a08aae admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:36.682091 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab67233-19d9-4468-94b9-c9e937a08aae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:36.684055 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab67233-19d9-4468-94b9-c9e937a08aae admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:36.708795 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1098/2205] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:36:36 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:36:36.713344 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ed627-ec65-4512-9416-23dec23280aa admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:36.726160 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ed627-ec65-4512-9416-23dec23280aa admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:36.731170 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d64a50-8028-46b9-9461-b7724aa916e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:36.731941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d64a50-8028-46b9-9461-b7724aa916e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:36.732522 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d64a50-8028-46b9-9461-b7724aa916e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:36.732904 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d64a50-8028-46b9-9461-b7724aa916e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:36.733773 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d64a50-8028-46b9-9461-b7724aa916e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:36.735509 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d64a50-8028-46b9-9461-b7724aa916e2 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:36.757302 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1108/2206] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:36:36.758824 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab67233-19d9-4468-94b9-c9e937a08aae admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:36.767674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:36.782382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab67233-19d9-4468-94b9-c9e937a08aae admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:36.788143 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc80ff5-e7d7-471a-9680-125bc7481f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:36.789278 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc80ff5-e7d7-471a-9680-125bc7481f26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:36.789642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc80ff5-e7d7-471a-9680-125bc7481f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:36.789987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc80ff5-e7d7-471a-9680-125bc7481f26 admin admin] PATH_INFO: `/v3/projects/bbbc2713d5854dd49b08622f859e1ce1/users/f0eec822d5ec487782e6b0015dc5f421/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:36.809502 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc80ff5-e7d7-471a-9680-125bc7481f26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbbc2713d5854dd49b08622f859e1ce1, user_id=f0eec822d5ec487782e6b0015dc5f421, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:36.813245 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc80ff5-e7d7-471a-9680-125bc7481f26 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:36.855794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1099/2207] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:36:36 2022] PUT /identity/v3/projects/bbbc2713d5854dd49b08622f859e1ce1/users/f0eec822d5ec487782e6b0015dc5f421/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:36.856596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d64a50-8028-46b9-9461-b7724aa916e2 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:36.872541 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d64a50-8028-46b9-9461-b7724aa916e2 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:36.980026 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50576e43-bde3-4115-bd01-19fddb1f8ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:36.981015 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50576e43-bde3-4115-bd01-19fddb1f8ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:36.983739 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50576e43-bde3-4115-bd01-19fddb1f8ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:36.984185 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50576e43-bde3-4115-bd01-19fddb1f8ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:36.985469 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50576e43-bde3-4115-bd01-19fddb1f8ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:36.987659 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50576e43-bde3-4115-bd01-19fddb1f8ca6 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:37.003205 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1109/2208] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:36:37.007476 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc80ff5-e7d7-471a-9680-125bc7481f26 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:37.020483 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc80ff5-e7d7-471a-9680-125bc7481f26 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:37.057536 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47878be4-0372-416b-9968-65bab287c1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:37.058470 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47878be4-0372-416b-9968-65bab287c1b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:37.058754 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47878be4-0372-416b-9968-65bab287c1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:37.059194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47878be4-0372-416b-9968-65bab287c1b0 admin admin] PATH_INFO: `/v3/projects/bbbc2713d5854dd49b08622f859e1ce1/users/f0eec822d5ec487782e6b0015dc5f421/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:37.063228 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47878be4-0372-416b-9968-65bab287c1b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbbc2713d5854dd49b08622f859e1ce1, user_id=f0eec822d5ec487782e6b0015dc5f421, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:37.066644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47878be4-0372-416b-9968-65bab287c1b0 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:37.081839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1100/2209] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:36:37 2022] PUT /identity/v3/projects/bbbc2713d5854dd49b08622f859e1ce1/users/f0eec822d5ec487782e6b0015dc5f421/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:40.166918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb1405-4126-48aa-8bb7-94c3bb3ee364 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:40.167627 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb1405-4126-48aa-8bb7-94c3bb3ee364 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:40.168028 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb1405-4126-48aa-8bb7-94c3bb3ee364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:40.229836 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-acdb1405-4126-48aa-8bb7-94c3bb3ee364 None None] MFA Rules not processed for user `cfc2f31f654e486d8505d80aad17d17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:40.231263 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-acdb1405-4126-48aa-8bb7-94c3bb3ee364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:40.460667 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1110/2210] 10.222.0.40 () {58 vars in 967 bytes} [Thu Jun 16 17:36:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:40.477487 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47878be4-0372-416b-9968-65bab287c1b0 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:40.479293 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-47878be4-0372-416b-9968-65bab287c1b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:40.622395 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47878be4-0372-416b-9968-65bab287c1b0 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:40.631190 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38572f7-5d31-4977-a77c-781f35fc102e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a37d1201d54afdad9b228ef4f2d787', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:40.632307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38572f7-5d31-4977-a77c-781f35fc102e service swift] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:40.632831 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38572f7-5d31-4977-a77c-781f35fc102e service swift] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:40.633267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38572f7-5d31-4977-a77c-781f35fc102e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:40.643935 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38572f7-5d31-4977-a77c-781f35fc102e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:40.651616 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38572f7-5d31-4977-a77c-781f35fc102e service swift] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:40.670240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1101/2211] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 17:36:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:36:43.959027 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb1405-4126-48aa-8bb7-94c3bb3ee364 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:43.961848 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-acdb1405-4126-48aa-8bb7-94c3bb3ee364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:44.084253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:44.139171 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb1405-4126-48aa-8bb7-94c3bb3ee364 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:44.152524 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76147b9c-0ced-4d10-949d-1a895fdbc36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:44.155107 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76147b9c-0ced-4d10-949d-1a895fdbc36d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:44.155746 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76147b9c-0ced-4d10-949d-1a895fdbc36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:44.157962 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76147b9c-0ced-4d10-949d-1a895fdbc36d admin admin] PATH_INFO: `/v3/users/305767c6f53e4239b8c2b209914065eb` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:44.162628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76147b9c-0ced-4d10-949d-1a895fdbc36d admin admin] RBAC: Authorizing `identity:delete_user(user_id=305767c6f53e4239b8c2b209914065eb)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:44.167484 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76147b9c-0ced-4d10-949d-1a895fdbc36d admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:44.270994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-76147b9c-0ced-4d10-949d-1a895fdbc36d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '305767c6f53e4239b8c2b209914065eb'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:44.296752 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-76147b9c-0ced-4d10-949d-1a895fdbc36d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '305767c6f53e4239b8c2b209914065eb'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:44.298227 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-76147b9c-0ced-4d10-949d-1a895fdbc36d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '305767c6f53e4239b8c2b209914065eb'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:44.315932 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-76147b9c-0ced-4d10-949d-1a895fdbc36d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '305767c6f53e4239b8c2b209914065eb'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:44.339064 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-76147b9c-0ced-4d10-949d-1a895fdbc36d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '305767c6f53e4239b8c2b209914065eb'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:44.364049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1111/2212] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:36:43 2022] DELETE /identity/v3/users/305767c6f53e4239b8c2b209914065eb => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:45.375157 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38572f7-5d31-4977-a77c-781f35fc102e service swift] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:45.390799 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38572f7-5d31-4977-a77c-781f35fc102e service swift] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:45.514325 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43acbad-3a23-43e1-8498-618d4ae87038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:45.516038 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43acbad-3a23-43e1-8498-618d4ae87038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:45.516390 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43acbad-3a23-43e1-8498-618d4ae87038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:45.516734 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43acbad-3a23-43e1-8498-618d4ae87038 admin admin] PATH_INFO: `/v3/projects/20bf4fe4ecc74735a2abdb5068b0785a` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:45.518870 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43acbad-3a23-43e1-8498-618d4ae87038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20bf4fe4ecc74735a2abdb5068b0785a)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:45.521715 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43acbad-3a23-43e1-8498-618d4ae87038 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:45.608656 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c43acbad-3a23-43e1-8498-618d4ae87038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20bf4fe4ecc74735a2abdb5068b0785a', 'request_id': 'req-c43acbad-3a23-43e1-8498-618d4ae87038'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:45.627107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c43acbad-3a23-43e1-8498-618d4ae87038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20bf4fe4ecc74735a2abdb5068b0785a', 'request_id': 'req-c43acbad-3a23-43e1-8498-618d4ae87038'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:45.629799 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:45.639954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c43acbad-3a23-43e1-8498-618d4ae87038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20bf4fe4ecc74735a2abdb5068b0785a', 'request_id': 'req-c43acbad-3a23-43e1-8498-618d4ae87038'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:45.660917 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c43acbad-3a23-43e1-8498-618d4ae87038 admin admin] The token cache is being invalidate because project 20bf4fe4ecc74735a2abdb5068b0785a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:36:45.661331 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c43acbad-3a23-43e1-8498-618d4ae87038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:45.662178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c43acbad-3a23-43e1-8498-618d4ae87038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:45.664177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1102/2213] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:36:45 2022] DELETE /identity/v3/projects/20bf4fe4ecc74735a2abdb5068b0785a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:45.671598 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5852581-da44-4532-980c-90d8dafbeb7e None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:45.672133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5852581-da44-4532-980c-90d8dafbeb7e None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:45.672133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5852581-da44-4532-980c-90d8dafbeb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:45.738296 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-b5852581-da44-4532-980c-90d8dafbeb7e None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:45.744295 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b5852581-da44-4532-980c-90d8dafbeb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:45.827661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1112/2214] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:45.829255 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616f729-91cc-46a5-962c-3ca17044b365 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:45.829555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616f729-91cc-46a5-962c-3ca17044b365 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:45.829839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616f729-91cc-46a5-962c-3ca17044b365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:45.850406 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d616f729-91cc-46a5-962c-3ca17044b365 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:45.851934 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d616f729-91cc-46a5-962c-3ca17044b365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:45.862978 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1103/2215] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:36:45.867267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5852581-da44-4532-980c-90d8dafbeb7e None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:45.882828 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5852581-da44-4532-980c-90d8dafbeb7e None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:45.890852 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd83401-f408-4680-838b-2da475182607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:45.890852 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd83401-f408-4680-838b-2da475182607 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:45.890852 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd83401-f408-4680-838b-2da475182607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:45.890852 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd83401-f408-4680-838b-2da475182607 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:45.891593 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd83401-f408-4680-838b-2da475182607 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:36:45.892071 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd83401-f408-4680-838b-2da475182607 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:45.893740 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd83401-f408-4680-838b-2da475182607 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:45.912487 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1113/2216] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:36:45.913570 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616f729-91cc-46a5-962c-3ca17044b365 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:45.927194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616f729-91cc-46a5-962c-3ca17044b365 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:45.932787 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4823e5-84d5-4680-ab8f-7340597273e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:45.935585 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4823e5-84d5-4680-ab8f-7340597273e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:45.936164 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4823e5-84d5-4680-ab8f-7340597273e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:45.936802 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4823e5-84d5-4680-ab8f-7340597273e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:45.937865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4823e5-84d5-4680-ab8f-7340597273e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:45.940205 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4823e5-84d5-4680-ab8f-7340597273e5 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:45.987609 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1104/2217] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:36:45 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:36:45.988646 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd83401-f408-4680-838b-2da475182607 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:46.010596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd83401-f408-4680-838b-2da475182607 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:46.012241 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686ced9-3653-43db-96c5-fe1499ecfd0d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:46.013761 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686ced9-3653-43db-96c5-fe1499ecfd0d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:46.013761 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686ced9-3653-43db-96c5-fe1499ecfd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:46.032365 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:46.084889 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1686ced9-3653-43db-96c5-fe1499ecfd0d None None] MFA Rules not processed for user `f0eec822d5ec487782e6b0015dc5f421`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:46.086163 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1686ced9-3653-43db-96c5-fe1499ecfd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:46.103710 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef5f83-285e-409e-9957-15a1244abf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:46.104696 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef5f83-285e-409e-9957-15a1244abf98 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:46.104994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef5f83-285e-409e-9957-15a1244abf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:46.105314 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef5f83-285e-409e-9957-15a1244abf98 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:46.106185 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef5f83-285e-409e-9957-15a1244abf98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:46.108738 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef5f83-285e-409e-9957-15a1244abf98 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:46.142739 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1114/2218] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:36:45 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:36:46.147011 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef5f83-285e-409e-9957-15a1244abf98 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:46.161219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef5f83-285e-409e-9957-15a1244abf98 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:46.166264 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c40cd8-7496-4672-9c56-1452965726fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:46.166958 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c40cd8-7496-4672-9c56-1452965726fb admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:46.167258 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c40cd8-7496-4672-9c56-1452965726fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:46.167909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c40cd8-7496-4672-9c56-1452965726fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:46.168684 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c40cd8-7496-4672-9c56-1452965726fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:46.171963 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c40cd8-7496-4672-9c56-1452965726fb admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:46.186747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1115/2219] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:36:46.190767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c40cd8-7496-4672-9c56-1452965726fb admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:46.208965 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c40cd8-7496-4672-9c56-1452965726fb admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:46.234433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb5f58-f542-4c41-a096-c447273c1f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:46.235141 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb5f58-f542-4c41-a096-c447273c1f90 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:46.235525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb5f58-f542-4c41-a096-c447273c1f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:46.235708 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb5f58-f542-4c41-a096-c447273c1f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:46.236469 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb5f58-f542-4c41-a096-c447273c1f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:46.240341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb5f58-f542-4c41-a096-c447273c1f90 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:46.258254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1116/2221] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:36:46.258439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1105/2221] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:36:46.258541 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb5f58-f542-4c41-a096-c447273c1f90 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:46.260641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686ced9-3653-43db-96c5-fe1499ecfd0d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:46.270492 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-7ecb5f58-f542-4c41-a096-c447273c1f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:46.296865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686ced9-3653-43db-96c5-fe1499ecfd0d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:46.320186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc9a97-ce09-42cb-8796-32218e88924e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:46.324609 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc9a97-ce09-42cb-8796-32218e88924e admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:46.324609 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc9a97-ce09-42cb-8796-32218e88924e admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:46.327196 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc9a97-ce09-42cb-8796-32218e88924e admin admin] PATH_INFO: `/v3/projects/35fdc8ef703e43f1a730a868721c4a2d/users/b41334c5088e4f4990637fd7e2964999/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:46.369395 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc9a97-ce09-42cb-8796-32218e88924e admin admin] RBAC: Authorizing `identity:create_grant(project_id=35fdc8ef703e43f1a730a868721c4a2d, user_id=b41334c5088e4f4990637fd7e2964999, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:46.373262 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc9a97-ce09-42cb-8796-32218e88924e admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:46.381967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb5f58-f542-4c41-a096-c447273c1f90 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:46.391266 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbde21-9e0d-4829-a6eb-41a253b550b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:46.392143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbde21-9e0d-4829-a6eb-41a253b550b3 service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:46.392608 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbde21-9e0d-4829-a6eb-41a253b550b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:46.393061 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbde21-9e0d-4829-a6eb-41a253b550b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:46.398271 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-3ebbde21-9e0d-4829-a6eb-41a253b550b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:46.408233 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1106/2222] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:36:46 2022] PUT /identity/v3/projects/35fdc8ef703e43f1a730a868721c4a2d/users/b41334c5088e4f4990637fd7e2964999/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:36:46.410781 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679de87-708c-46cc-ba97-33bc84b00000 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:46.411240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679de87-708c-46cc-ba97-33bc84b00000 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:46.411550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679de87-708c-46cc-ba97-33bc84b00000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:46.446232 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0679de87-708c-46cc-ba97-33bc84b00000 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:46.447747 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0679de87-708c-46cc-ba97-33bc84b00000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:46.482250 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbde21-9e0d-4829-a6eb-41a253b550b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:46.484406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbde21-9e0d-4829-a6eb-41a253b550b3 service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:46.495964 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1117/2223] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:46.547464 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1107/2224] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:46.558981 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbde21-9e0d-4829-a6eb-41a253b550b3 service nova] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:46.561187 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-3ebbde21-9e0d-4829-a6eb-41a253b550b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:46.611673 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:46.672787 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbde21-9e0d-4829-a6eb-41a253b550b3 service nova] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:46.742436 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1268599-c436-473b-be7d-3a9c99ff4798 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:46.743767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1268599-c436-473b-be7d-3a9c99ff4798 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:46.744157 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1268599-c436-473b-be7d-3a9c99ff4798 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:46.744460 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1268599-c436-473b-be7d-3a9c99ff4798 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:46.755518 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1268599-c436-473b-be7d-3a9c99ff4798 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:46.757115 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1268599-c436-473b-be7d-3a9c99ff4798 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:46.775245 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1118/2225] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:53.215383 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0227830-d413-450b-8ed9-3d5885c938f5 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:53.216034 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0227830-d413-450b-8ed9-3d5885c938f5 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:53.216136 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0227830-d413-450b-8ed9-3d5885c938f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:53.241240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-f0227830-d413-450b-8ed9-3d5885c938f5 None None] MFA Rules not processed for user `a5c37b79bf3d40f4a6182cc296585c18`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:53.242889 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f0227830-d413-450b-8ed9-3d5885c938f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:53.269652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:53.353297 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1108/2226] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:53.657279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03b970-96e1-4c99-b371-0820c5e76f71 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:53.658343 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03b970-96e1-4c99-b371-0820c5e76f71 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:53.658824 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03b970-96e1-4c99-b371-0820c5e76f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:53.693492 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-3f03b970-96e1-4c99-b371-0820c5e76f71 None None] MFA Rules not processed for user `b41334c5088e4f4990637fd7e2964999`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:53.694907 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-3f03b970-96e1-4c99-b371-0820c5e76f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:53.801309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1119/2227] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:53.804763 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0227830-d413-450b-8ed9-3d5885c938f5 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:53.823461 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0227830-d413-450b-8ed9-3d5885c938f5 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:53.829848 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0096db-b384-4273-95e5-69af84404525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:53.830536 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0096db-b384-4273-95e5-69af84404525 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:53.830825 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0096db-b384-4273-95e5-69af84404525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:53.831267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0096db-b384-4273-95e5-69af84404525 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:53.832068 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0096db-b384-4273-95e5-69af84404525 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:53.834370 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0096db-b384-4273-95e5-69af84404525 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:53.869291 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1109/2228] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:36:53 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:36:53.873261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03b970-96e1-4c99-b371-0820c5e76f71 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:53.885485 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03b970-96e1-4c99-b371-0820c5e76f71 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:53.971404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:53.982737 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901386d-1517-4d53-9f5e-8354e7174491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:53.983443 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901386d-1517-4d53-9f5e-8354e7174491 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:53.983735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901386d-1517-4d53-9f5e-8354e7174491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:53.984016 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901386d-1517-4d53-9f5e-8354e7174491 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:53.984745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7901386d-1517-4d53-9f5e-8354e7174491 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:53.986904 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7901386d-1517-4d53-9f5e-8354e7174491 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:54.017738 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1120/2229] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:36:53 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:36:54.018995 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0096db-b384-4273-95e5-69af84404525 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:54.030189 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0096db-b384-4273-95e5-69af84404525 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:54.040528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a0eee-7fcc-43ce-ad00-859024249b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:54.041262 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a0eee-7fcc-43ce-ad00-859024249b5a admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:54.041548 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a0eee-7fcc-43ce-ad00-859024249b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:54.041824 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a0eee-7fcc-43ce-ad00-859024249b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:54.042560 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a0eee-7fcc-43ce-ad00-859024249b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:54.044831 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a0eee-7fcc-43ce-ad00-859024249b5a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:54.063237 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1110/2230] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:36:54.065185 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901386d-1517-4d53-9f5e-8354e7174491 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:54.081898 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901386d-1517-4d53-9f5e-8354e7174491 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:54.087289 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff65f01-588b-45cc-8f50-63a3c81f4f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:54.088267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff65f01-588b-45cc-8f50-63a3c81f4f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:54.088586 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff65f01-588b-45cc-8f50-63a3c81f4f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:54.089068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff65f01-588b-45cc-8f50-63a3c81f4f1a admin admin] PATH_INFO: `/v3/projects/119182b448df4b0982dcd69db60fd7a6/users/1c021dd109504201be15e79f3c07378e/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:54.107812 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff65f01-588b-45cc-8f50-63a3c81f4f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=119182b448df4b0982dcd69db60fd7a6, user_id=1c021dd109504201be15e79f3c07378e, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:54.110335 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff65f01-588b-45cc-8f50-63a3c81f4f1a admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:54.129960 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1121/2231] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:36:54 2022] PUT /identity/v3/projects/119182b448df4b0982dcd69db60fd7a6/users/1c021dd109504201be15e79f3c07378e/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:36:54.133914 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a0eee-7fcc-43ce-ad00-859024249b5a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:54.145687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a0eee-7fcc-43ce-ad00-859024249b5a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:54.315466 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19ef14-39f7-4b47-bebd-84f8f50bdb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:54.316390 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19ef14-39f7-4b47-bebd-84f8f50bdb98 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:54.316745 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19ef14-39f7-4b47-bebd-84f8f50bdb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:54.316976 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19ef14-39f7-4b47-bebd-84f8f50bdb98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:54.317829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d19ef14-39f7-4b47-bebd-84f8f50bdb98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:54.319585 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d19ef14-39f7-4b47-bebd-84f8f50bdb98 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:54.339533 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1111/2232] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:36:54.341425 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff65f01-588b-45cc-8f50-63a3c81f4f1a admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:54.354484 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff65f01-588b-45cc-8f50-63a3c81f4f1a admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:54.361691 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff56f49-b2a2-4f0b-9436-b1e4a3f2950a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:54.362747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff56f49-b2a2-4f0b-9436-b1e4a3f2950a admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:54.363102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff56f49-b2a2-4f0b-9436-b1e4a3f2950a admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:54.363483 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff56f49-b2a2-4f0b-9436-b1e4a3f2950a admin admin] PATH_INFO: `/v3/projects/119182b448df4b0982dcd69db60fd7a6/users/1c021dd109504201be15e79f3c07378e/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:54.370555 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff56f49-b2a2-4f0b-9436-b1e4a3f2950a admin admin] RBAC: Authorizing `identity:create_grant(project_id=119182b448df4b0982dcd69db60fd7a6, user_id=1c021dd109504201be15e79f3c07378e, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:54.374963 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff56f49-b2a2-4f0b-9436-b1e4a3f2950a admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:54.397073 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1122/2233] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:36:54 2022] PUT /identity/v3/projects/119182b448df4b0982dcd69db60fd7a6/users/1c021dd109504201be15e79f3c07378e/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:55.478156 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19ef14-39f7-4b47-bebd-84f8f50bdb98 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:55.482503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5d19ef14-39f7-4b47-bebd-84f8f50bdb98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:55.503213 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19ef14-39f7-4b47-bebd-84f8f50bdb98 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:55.624391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6745ba0-506f-4b39-b990-864acc45f067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:55.625562 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6745ba0-506f-4b39-b990-864acc45f067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:55.626108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6745ba0-506f-4b39-b990-864acc45f067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:55.626624 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6745ba0-506f-4b39-b990-864acc45f067 admin admin] PATH_INFO: `/v3/users/1793a3b352da4b43acf698aa5bbce3fe` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:55.628186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6745ba0-506f-4b39-b990-864acc45f067 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1793a3b352da4b43acf698aa5bbce3fe)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:55.630839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6745ba0-506f-4b39-b990-864acc45f067 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:55.666260 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:55.708569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d6745ba0-506f-4b39-b990-864acc45f067 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1793a3b352da4b43acf698aa5bbce3fe'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:55.724384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d6745ba0-506f-4b39-b990-864acc45f067 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1793a3b352da4b43acf698aa5bbce3fe'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:55.726104 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d6745ba0-506f-4b39-b990-864acc45f067 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1793a3b352da4b43acf698aa5bbce3fe'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:55.739603 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d6745ba0-506f-4b39-b990-864acc45f067 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1793a3b352da4b43acf698aa5bbce3fe'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:55.750649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d6745ba0-506f-4b39-b990-864acc45f067 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1793a3b352da4b43acf698aa5bbce3fe'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:55.765360 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1112/2234] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:36:55 2022] DELETE /identity/v3/users/1793a3b352da4b43acf698aa5bbce3fe => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:55.770963 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff7359-45c6-45f1-bd85-0a05c7d1f9a9 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:55.771697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff7359-45c6-45f1-bd85-0a05c7d1f9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:55.772797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff7359-45c6-45f1-bd85-0a05c7d1f9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:55.797908 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-f3ff7359-45c6-45f1-bd85-0a05c7d1f9a9 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:55.799128 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-f3ff7359-45c6-45f1-bd85-0a05c7d1f9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:55.917597 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1123/2235] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:36:55.958885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6745ba0-506f-4b39-b990-864acc45f067 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:55.970214 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d6745ba0-506f-4b39-b990-864acc45f067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:56.131190 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6745ba0-506f-4b39-b990-864acc45f067 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:56.135580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901e744-19f6-47ef-b49d-ec271d292d0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:56.136444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901e744-19f6-47ef-b49d-ec271d292d0b service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:56.136921 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901e744-19f6-47ef-b49d-ec271d292d0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:56.138074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901e744-19f6-47ef-b49d-ec271d292d0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:56.151258 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6901e744-19f6-47ef-b49d-ec271d292d0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:56.162034 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6901e744-19f6-47ef-b49d-ec271d292d0b service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:56.178018 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1113/2236] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:57.624311 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff7359-45c6-45f1-bd85-0a05c7d1f9a9 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:57.626469 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-f3ff7359-45c6-45f1-bd85-0a05c7d1f9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:57.679776 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff7359-45c6-45f1-bd85-0a05c7d1f9a9 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:57.707949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629b4bd-1c2e-48c7-a1db-050323877b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:57.708959 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629b4bd-1c2e-48c7-a1db-050323877b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:57.709272 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629b4bd-1c2e-48c7-a1db-050323877b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:57.709784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629b4bd-1c2e-48c7-a1db-050323877b80 admin admin] PATH_INFO: `/v3/projects/c5c9deabd6444227b04c98d0eb1b3679` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:57.716910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629b4bd-1c2e-48c7-a1db-050323877b80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5c9deabd6444227b04c98d0eb1b3679)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:57.720080 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629b4bd-1c2e-48c7-a1db-050323877b80 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:57.803975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-8629b4bd-1c2e-48c7-a1db-050323877b80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5c9deabd6444227b04c98d0eb1b3679', 'request_id': 'req-8629b4bd-1c2e-48c7-a1db-050323877b80'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:57.813449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-8629b4bd-1c2e-48c7-a1db-050323877b80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5c9deabd6444227b04c98d0eb1b3679', 'request_id': 'req-8629b4bd-1c2e-48c7-a1db-050323877b80'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:57.822871 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-8629b4bd-1c2e-48c7-a1db-050323877b80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5c9deabd6444227b04c98d0eb1b3679', 'request_id': 'req-8629b4bd-1c2e-48c7-a1db-050323877b80'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:57.841842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-8629b4bd-1c2e-48c7-a1db-050323877b80 admin admin] The token cache is being invalidate because project c5c9deabd6444227b04c98d0eb1b3679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:36:57.842298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-8629b4bd-1c2e-48c7-a1db-050323877b80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:57.843055 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-8629b4bd-1c2e-48c7-a1db-050323877b80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:57.845032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1124/2237] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:36:57 2022] DELETE /identity/v3/projects/c5c9deabd6444227b04c98d0eb1b3679 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:57.850052 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc053072-af71-4ae4-a438-61f2c91e71a3 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:57.850534 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc053072-af71-4ae4-a438-61f2c91e71a3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:57.850642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc053072-af71-4ae4-a438-61f2c91e71a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:57.891462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-dc053072-af71-4ae4-a438-61f2c91e71a3 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:57.892973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-dc053072-af71-4ae4-a438-61f2c91e71a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:58.004306 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f0830-d423-4cd2-96d9-44173f10d084 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:58.004610 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f0830-d423-4cd2-96d9-44173f10d084 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:58.004908 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f0830-d423-4cd2-96d9-44173f10d084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:58.032423 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1114/2238] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:58.046639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-332f0830-d423-4cd2-96d9-44173f10d084 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:58.048382 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-332f0830-d423-4cd2-96d9-44173f10d084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:58.096201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1125/2239] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:36:58.099538 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc053072-af71-4ae4-a438-61f2c91e71a3 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:58.120698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc053072-af71-4ae4-a438-61f2c91e71a3 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:58.136318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4547eaf-b518-4176-bfbf-e309b86a50bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:58.137298 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4547eaf-b518-4176-bfbf-e309b86a50bd admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:58.137487 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4547eaf-b518-4176-bfbf-e309b86a50bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:58.137760 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4547eaf-b518-4176-bfbf-e309b86a50bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:58.153515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4547eaf-b518-4176-bfbf-e309b86a50bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:36:58.154667 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4547eaf-b518-4176-bfbf-e309b86a50bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:58.156530 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4547eaf-b518-4176-bfbf-e309b86a50bd admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:58.193678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1115/2240] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:36:58.204517 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f0830-d423-4cd2-96d9-44173f10d084 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:58.218979 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f0830-d423-4cd2-96d9-44173f10d084 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:58.224438 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c49c9e1-fefc-41e1-8134-6e1a336f9ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:58.226019 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49c9e1-fefc-41e1-8134-6e1a336f9ba9 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:58.226598 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49c9e1-fefc-41e1-8134-6e1a336f9ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:58.227093 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49c9e1-fefc-41e1-8134-6e1a336f9ba9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:58.228648 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c49c9e1-fefc-41e1-8134-6e1a336f9ba9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:58.231329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c49c9e1-fefc-41e1-8134-6e1a336f9ba9 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:58.294925 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1126/2241] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:36:58 2022] POST /identity/v3/projects => generated 389 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:36:58.299857 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4547eaf-b518-4176-bfbf-e309b86a50bd admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:58.318021 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4547eaf-b518-4176-bfbf-e309b86a50bd admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:58.391275 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:58.489785 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb11580-7a5a-4d4b-a762-419df9874073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:58.490491 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb11580-7a5a-4d4b-a762-419df9874073 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:58.490782 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb11580-7a5a-4d4b-a762-419df9874073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:58.491096 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb11580-7a5a-4d4b-a762-419df9874073 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:58.491842 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb11580-7a5a-4d4b-a762-419df9874073 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:58.494426 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb11580-7a5a-4d4b-a762-419df9874073 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:58.555467 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1116/2242] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:36:58 2022] POST /identity/v3/users => generated 295 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:36:58.561216 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c49c9e1-fefc-41e1-8134-6e1a336f9ba9 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:58.570595 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:58.576450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c49c9e1-fefc-41e1-8134-6e1a336f9ba9 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:58.586646 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c71bc-937a-4cc2-b661-867add16d3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:58.588023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c71bc-937a-4cc2-b661-867add16d3ac admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:58.588023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c71bc-937a-4cc2-b661-867add16d3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:58.588383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c71bc-937a-4cc2-b661-867add16d3ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:58.589345 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c71bc-937a-4cc2-b661-867add16d3ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:58.591999 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c71bc-937a-4cc2-b661-867add16d3ac admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:58.624059 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1127/2243] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:36:58.628346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb11580-7a5a-4d4b-a762-419df9874073 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:58.641770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb11580-7a5a-4d4b-a762-419df9874073 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:58.655959 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda8065-15b9-4fc8-968c-e4fbc8f9f547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:58.657004 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda8065-15b9-4fc8-968c-e4fbc8f9f547 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:58.658451 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda8065-15b9-4fc8-968c-e4fbc8f9f547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:58.658675 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda8065-15b9-4fc8-968c-e4fbc8f9f547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:58.661871 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda8065-15b9-4fc8-968c-e4fbc8f9f547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:58.663760 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda8065-15b9-4fc8-968c-e4fbc8f9f547 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:58.690491 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1117/2244] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:36:58.695191 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c71bc-937a-4cc2-b661-867add16d3ac admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:58.709701 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c71bc-937a-4cc2-b661-867add16d3ac admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:58.721924 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09620a-9408-4e40-9319-ed40fdc0ef68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:58.724691 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09620a-9408-4e40-9319-ed40fdc0ef68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:58.726636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09620a-9408-4e40-9319-ed40fdc0ef68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:58.727636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09620a-9408-4e40-9319-ed40fdc0ef68 admin admin] PATH_INFO: `/v3/projects/592df0bca489405788b1d79bf6abd33b/users/a6659b0eb5f0467aad81824c0a66cf78/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:58.772905 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09620a-9408-4e40-9319-ed40fdc0ef68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=592df0bca489405788b1d79bf6abd33b, user_id=a6659b0eb5f0467aad81824c0a66cf78, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:58.779641 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09620a-9408-4e40-9319-ed40fdc0ef68 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:58.811759 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1128/2245] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:36:58 2022] PUT /identity/v3/projects/592df0bca489405788b1d79bf6abd33b/users/a6659b0eb5f0467aad81824c0a66cf78/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:58.822366 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af5c58-a03d-4f18-ab48-cb361cb13de4 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:58.822679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af5c58-a03d-4f18-ab48-cb361cb13de4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:58.823122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af5c58-a03d-4f18-ab48-cb361cb13de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:58.898393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d2af5c58-a03d-4f18-ab48-cb361cb13de4 None None] MFA Rules not processed for user `a6659b0eb5f0467aad81824c0a66cf78`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:58.906185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d2af5c58-a03d-4f18-ab48-cb361cb13de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:59.099830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1118/2246] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:36:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:59.125521 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09620a-9408-4e40-9319-ed40fdc0ef68 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:59.128782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-2d09620a-9408-4e40-9319-ed40fdc0ef68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:59.295145 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09620a-9408-4e40-9319-ed40fdc0ef68 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:59.313854 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9560063-7a88-4974-9114-8be62b723e0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:59.314668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9560063-7a88-4974-9114-8be62b723e0e service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:59.316538 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9560063-7a88-4974-9114-8be62b723e0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:59.317054 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9560063-7a88-4974-9114-8be62b723e0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:59.344557 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9560063-7a88-4974-9114-8be62b723e0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:59.346334 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9560063-7a88-4974-9114-8be62b723e0e service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:59.370412 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1129/2247] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:03.345636 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26f57f-8a85-4f26-a935-785eec7360d8 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:03.346191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26f57f-8a85-4f26-a935-785eec7360d8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:03.346191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26f57f-8a85-4f26-a935-785eec7360d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:03.411637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-5f26f57f-8a85-4f26-a935-785eec7360d8 None None] MFA Rules not processed for user `1c021dd109504201be15e79f3c07378e`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:03.413383 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5f26f57f-8a85-4f26-a935-785eec7360d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:03.505359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1119/2248] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:37:03 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:03.518465 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9560063-7a88-4974-9114-8be62b723e0e service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:03.520053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-c9560063-7a88-4974-9114-8be62b723e0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:03.648705 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9560063-7a88-4974-9114-8be62b723e0e service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:03.655726 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931e4ed-0d3f-46e4-b4cd-711a29c1f37e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:03.656406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931e4ed-0d3f-46e4-b4cd-711a29c1f37e service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:03.656703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931e4ed-0d3f-46e4-b4cd-711a29c1f37e service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:03.656992 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931e4ed-0d3f-46e4-b4cd-711a29c1f37e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:03.658905 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-f931e4ed-0d3f-46e4-b4cd-711a29c1f37e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:03.709652 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:37:03.729581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f931e4ed-0d3f-46e4-b4cd-711a29c1f37e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:03.731903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f931e4ed-0d3f-46e4-b4cd-711a29c1f37e service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:03.741569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1130/2249] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:04.847803 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62dd9a0-9166-4923-95fc-2fababf44fb1 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:04.848332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62dd9a0-9166-4923-95fc-2fababf44fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:04.848332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62dd9a0-9166-4923-95fc-2fababf44fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:04.866581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a62dd9a0-9166-4923-95fc-2fababf44fb1 None None] MFA Rules not processed for user `b41334c5088e4f4990637fd7e2964999`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:04.868038 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a62dd9a0-9166-4923-95fc-2fababf44fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:04.875322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1120/2250] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:37:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:13.737166 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=8 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:37:14.861386 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:37:15.389354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbeb24f-afde-48fb-b601-b75b5fc940ad None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:15.389907 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbeb24f-afde-48fb-b601-b75b5fc940ad None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:15.389907 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbeb24f-afde-48fb-b601-b75b5fc940ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:15.410512 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-bcbeb24f-afde-48fb-b601-b75b5fc940ad None None] MFA Rules not processed for user `faf0a0102da24cedaedd0c67273e275b`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:15.410512 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-bcbeb24f-afde-48fb-b601-b75b5fc940ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:15.470536 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1131/2251] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:37:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:19.606711 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62dd9a0-9166-4923-95fc-2fababf44fb1 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:19.608912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a62dd9a0-9166-4923-95fc-2fababf44fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:19.733688 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62dd9a0-9166-4923-95fc-2fababf44fb1 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:19.740487 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784744f-a545-4037-b488-5fff995b4034 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75f0586a8c1941448d750918183f6dc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:19.741800 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784744f-a545-4037-b488-5fff995b4034 service placement] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:19.742584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784744f-a545-4037-b488-5fff995b4034 service placement] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:19.743094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784744f-a545-4037-b488-5fff995b4034 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:19.745122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6784744f-a545-4037-b488-5fff995b4034 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:19.762561 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784744f-a545-4037-b488-5fff995b4034 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:19.764596 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784744f-a545-4037-b488-5fff995b4034 service placement] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:19.777880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1121/2252] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:19.921297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbeb24f-afde-48fb-b601-b75b5fc940ad None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:20.024326 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbeb24f-afde-48fb-b601-b75b5fc940ad None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:20.029831 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53793de2-a917-49dc-ae41-9467a831deeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:20.031091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53793de2-a917-49dc-ae41-9467a831deeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:20.031454 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53793de2-a917-49dc-ae41-9467a831deeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:20.031812 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53793de2-a917-49dc-ae41-9467a831deeb admin admin] PATH_INFO: `/v3/users/a6659b0eb5f0467aad81824c0a66cf78` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:20.033326 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53793de2-a917-49dc-ae41-9467a831deeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6659b0eb5f0467aad81824c0a66cf78)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:20.036910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53793de2-a917-49dc-ae41-9467a831deeb admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:20.135726 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-53793de2-a917-49dc-ae41-9467a831deeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6659b0eb5f0467aad81824c0a66cf78'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:20.146936 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-53793de2-a917-49dc-ae41-9467a831deeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6659b0eb5f0467aad81824c0a66cf78'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:20.148087 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-53793de2-a917-49dc-ae41-9467a831deeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6659b0eb5f0467aad81824c0a66cf78'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:20.158882 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-53793de2-a917-49dc-ae41-9467a831deeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6659b0eb5f0467aad81824c0a66cf78'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:20.179462 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-53793de2-a917-49dc-ae41-9467a831deeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6659b0eb5f0467aad81824c0a66cf78'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:20.186452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1132/2253] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:37:19 2022] DELETE /identity/v3/users/a6659b0eb5f0467aad81824c0a66cf78 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:37:20.191100 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300e32c-3306-4723-a62a-a5d37d3bda5c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:20.191612 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300e32c-3306-4723-a62a-a5d37d3bda5c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:20.191900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300e32c-3306-4723-a62a-a5d37d3bda5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:20.226237 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-c300e32c-3306-4723-a62a-a5d37d3bda5c None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:20.227478 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c300e32c-3306-4723-a62a-a5d37d3bda5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:20.305511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1122/2254] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:20.346408 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53793de2-a917-49dc-ae41-9467a831deeb admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:20.348381 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-53793de2-a917-49dc-ae41-9467a831deeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:20.388377 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:37:20.432040 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53793de2-a917-49dc-ae41-9467a831deeb admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:20.447288 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9cada-09bd-464b-87ff-044c8cbe14b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:20.448226 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9cada-09bd-464b-87ff-044c8cbe14b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:20.448857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9cada-09bd-464b-87ff-044c8cbe14b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:20.449780 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9cada-09bd-464b-87ff-044c8cbe14b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:20.468738 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e9cada-09bd-464b-87ff-044c8cbe14b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:20.470414 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e9cada-09bd-464b-87ff-044c8cbe14b7 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:20.484114 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1133/2255] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:37:20.956353 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300e32c-3306-4723-a62a-a5d37d3bda5c None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:20.958948 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c300e32c-3306-4723-a62a-a5d37d3bda5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:20.983271 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300e32c-3306-4723-a62a-a5d37d3bda5c None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:20.991092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baef0fd-cfa5-4b83-83e0-9fb856fce74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:20.992980 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baef0fd-cfa5-4b83-83e0-9fb856fce74a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:20.993988 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baef0fd-cfa5-4b83-83e0-9fb856fce74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:20.994484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baef0fd-cfa5-4b83-83e0-9fb856fce74a admin admin] PATH_INFO: `/v3/projects/592df0bca489405788b1d79bf6abd33b` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:20.997587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baef0fd-cfa5-4b83-83e0-9fb856fce74a admin admin] RBAC: Authorizing `identity:delete_project(project_id=592df0bca489405788b1d79bf6abd33b)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:21.000553 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baef0fd-cfa5-4b83-83e0-9fb856fce74a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:21.104587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8baef0fd-cfa5-4b83-83e0-9fb856fce74a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '592df0bca489405788b1d79bf6abd33b', 'request_id': 'req-8baef0fd-cfa5-4b83-83e0-9fb856fce74a'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:21.118776 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8baef0fd-cfa5-4b83-83e0-9fb856fce74a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '592df0bca489405788b1d79bf6abd33b', 'request_id': 'req-8baef0fd-cfa5-4b83-83e0-9fb856fce74a'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:21.127583 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8baef0fd-cfa5-4b83-83e0-9fb856fce74a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '592df0bca489405788b1d79bf6abd33b', 'request_id': 'req-8baef0fd-cfa5-4b83-83e0-9fb856fce74a'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:21.133003 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:37:21.142763 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8baef0fd-cfa5-4b83-83e0-9fb856fce74a admin admin] The token cache is being invalidate because project 592df0bca489405788b1d79bf6abd33b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:37:21.143461 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8baef0fd-cfa5-4b83-83e0-9fb856fce74a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:21.144378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-8baef0fd-cfa5-4b83-83e0-9fb856fce74a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:21.149508 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1123/2256] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:37:20 2022] DELETE /identity/v3/projects/592df0bca489405788b1d79bf6abd33b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:37:21.152485 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661fc33-c6dc-4a28-b114-b7c673346897 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:21.152999 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661fc33-c6dc-4a28-b114-b7c673346897 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:21.153109 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661fc33-c6dc-4a28-b114-b7c673346897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:21.172811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-2661fc33-c6dc-4a28-b114-b7c673346897 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:21.174375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-2661fc33-c6dc-4a28-b114-b7c673346897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:21.257372 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1134/2257] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:21.266076 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e27e7-80c1-486f-ab86-85dfc522d39d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:21.266441 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e27e7-80c1-486f-ab86-85dfc522d39d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:21.266979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e27e7-80c1-486f-ab86-85dfc522d39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:21.288486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-6d1e27e7-80c1-486f-ab86-85dfc522d39d None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:21.289890 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6d1e27e7-80c1-486f-ab86-85dfc522d39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:21.306755 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1124/2258] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:21.312268 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661fc33-c6dc-4a28-b114-b7c673346897 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:21.322867 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661fc33-c6dc-4a28-b114-b7c673346897 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:21.332134 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276496bb-b7e2-4632-b975-8b237c4f1e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:21.332922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276496bb-b7e2-4632-b975-8b237c4f1e78 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:21.335162 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276496bb-b7e2-4632-b975-8b237c4f1e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:21.335531 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276496bb-b7e2-4632-b975-8b237c4f1e78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:21.336402 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276496bb-b7e2-4632-b975-8b237c4f1e78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:37:21.336769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276496bb-b7e2-4632-b975-8b237c4f1e78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:21.338905 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276496bb-b7e2-4632-b975-8b237c4f1e78 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:21.355930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1135/2259] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:37:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:37:21.360301 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e27e7-80c1-486f-ab86-85dfc522d39d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:21.371680 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e27e7-80c1-486f-ab86-85dfc522d39d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:21.376849 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4388fe-db2a-4345-b3d9-06914fc6a56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:21.377710 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4388fe-db2a-4345-b3d9-06914fc6a56d admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:21.378099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4388fe-db2a-4345-b3d9-06914fc6a56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:21.378482 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4388fe-db2a-4345-b3d9-06914fc6a56d admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:21.379305 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4388fe-db2a-4345-b3d9-06914fc6a56d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:21.381389 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4388fe-db2a-4345-b3d9-06914fc6a56d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:21.415409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1125/2260] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:37:21 2022] POST /identity/v3/projects => generated 397 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:37:21.420490 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276496bb-b7e2-4632-b975-8b237c4f1e78 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:21.431123 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276496bb-b7e2-4632-b975-8b237c4f1e78 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:21.501387 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef236f-51b2-42cd-8faa-c19334dbcf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:21.502332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef236f-51b2-42cd-8faa-c19334dbcf35 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:21.502625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef236f-51b2-42cd-8faa-c19334dbcf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:21.502914 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef236f-51b2-42cd-8faa-c19334dbcf35 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:21.503830 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef236f-51b2-42cd-8faa-c19334dbcf35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:21.505605 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef236f-51b2-42cd-8faa-c19334dbcf35 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:21.537133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1136/2261] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:37:21 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:37:21.541029 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4388fe-db2a-4345-b3d9-06914fc6a56d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:21.554713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4388fe-db2a-4345-b3d9-06914fc6a56d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:21.560986 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208540c3-77fd-4bd2-bc5a-4f6d2dcdb0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:21.561838 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208540c3-77fd-4bd2-bc5a-4f6d2dcdb0af admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:21.562201 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208540c3-77fd-4bd2-bc5a-4f6d2dcdb0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:21.562547 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208540c3-77fd-4bd2-bc5a-4f6d2dcdb0af admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:21.563364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208540c3-77fd-4bd2-bc5a-4f6d2dcdb0af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:21.565090 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208540c3-77fd-4bd2-bc5a-4f6d2dcdb0af admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:21.578067 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1126/2262] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:37:21.582060 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef236f-51b2-42cd-8faa-c19334dbcf35 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:21.593636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef236f-51b2-42cd-8faa-c19334dbcf35 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:21.598360 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd29e6-fcb7-49dd-8f66-61d4de9effd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:21.599700 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd29e6-fcb7-49dd-8f66-61d4de9effd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:21.600020 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd29e6-fcb7-49dd-8f66-61d4de9effd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:21.600275 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd29e6-fcb7-49dd-8f66-61d4de9effd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:21.601027 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd29e6-fcb7-49dd-8f66-61d4de9effd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:21.602754 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd29e6-fcb7-49dd-8f66-61d4de9effd2 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:21.615066 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1137/2263] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:37:21.619835 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208540c3-77fd-4bd2-bc5a-4f6d2dcdb0af admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:21.631084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208540c3-77fd-4bd2-bc5a-4f6d2dcdb0af admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:21.636700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a270459-3cac-4d65-a9d5-8782fa09e2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:21.637565 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a270459-3cac-4d65-a9d5-8782fa09e2bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:21.638012 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a270459-3cac-4d65-a9d5-8782fa09e2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:21.638328 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a270459-3cac-4d65-a9d5-8782fa09e2bc admin admin] PATH_INFO: `/v3/projects/750141c5896f4002a4e15bbc892c6c55/users/f8b65ef2df7d46048c4f5669c1031a22/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:21.656361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a270459-3cac-4d65-a9d5-8782fa09e2bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=750141c5896f4002a4e15bbc892c6c55, user_id=f8b65ef2df7d46048c4f5669c1031a22, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:21.659294 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a270459-3cac-4d65-a9d5-8782fa09e2bc admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:21.678916 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1127/2264] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:37:21 2022] PUT /identity/v3/projects/750141c5896f4002a4e15bbc892c6c55/users/f8b65ef2df7d46048c4f5669c1031a22/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:37:21.689967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55df26-5fb4-428d-b6b0-212a7ab5e546 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:21.690285 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55df26-5fb4-428d-b6b0-212a7ab5e546 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:21.690583 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55df26-5fb4-428d-b6b0-212a7ab5e546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:21.724732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-7c55df26-5fb4-428d-b6b0-212a7ab5e546 None None] MFA Rules not processed for user `f8b65ef2df7d46048c4f5669c1031a22`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:21.726016 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-7c55df26-5fb4-428d-b6b0-212a7ab5e546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:21.757886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:37:21.807130 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1138/2265] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:37:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:21.819410 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a270459-3cac-4d65-a9d5-8782fa09e2bc admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:21.821187 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5a270459-3cac-4d65-a9d5-8782fa09e2bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:21.924181 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a270459-3cac-4d65-a9d5-8782fa09e2bc admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:21.928113 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fab8f8-529d-4e2d-a79d-301aea6e204d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:21.928795 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fab8f8-529d-4e2d-a79d-301aea6e204d service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:21.929083 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fab8f8-529d-4e2d-a79d-301aea6e204d service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:21.929419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fab8f8-529d-4e2d-a79d-301aea6e204d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:21.940072 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fab8f8-529d-4e2d-a79d-301aea6e204d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:21.941821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fab8f8-529d-4e2d-a79d-301aea6e204d service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:21.953437 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1128/2266] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:21 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:30.234741 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55df26-5fb4-428d-b6b0-212a7ab5e546 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:30.302562 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55df26-5fb4-428d-b6b0-212a7ab5e546 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:30.306978 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bf599-c6be-4a3c-8660-16e2606acbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:30.308366 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bf599-c6be-4a3c-8660-16e2606acbc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:30.308644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bf599-c6be-4a3c-8660-16e2606acbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:30.308927 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bf599-c6be-4a3c-8660-16e2606acbc9 admin admin] PATH_INFO: `/v3/users/f8b65ef2df7d46048c4f5669c1031a22` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:30.310169 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656bf599-c6be-4a3c-8660-16e2606acbc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8b65ef2df7d46048c4f5669c1031a22)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:30.312084 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656bf599-c6be-4a3c-8660-16e2606acbc9 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:30.362071 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-656bf599-c6be-4a3c-8660-16e2606acbc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8b65ef2df7d46048c4f5669c1031a22'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:30.369272 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-656bf599-c6be-4a3c-8660-16e2606acbc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8b65ef2df7d46048c4f5669c1031a22'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:30.370434 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-656bf599-c6be-4a3c-8660-16e2606acbc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8b65ef2df7d46048c4f5669c1031a22'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:30.379721 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-656bf599-c6be-4a3c-8660-16e2606acbc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8b65ef2df7d46048c4f5669c1031a22'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:30.396738 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-656bf599-c6be-4a3c-8660-16e2606acbc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8b65ef2df7d46048c4f5669c1031a22'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:30.406575 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1139/2267] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:37:30 2022] DELETE /identity/v3/users/f8b65ef2df7d46048c4f5669c1031a22 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:37:30.411939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9a5f7-8713-4b15-a71d-a10c206679c9 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:30.412793 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9a5f7-8713-4b15-a71d-a10c206679c9 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:30.413083 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9a5f7-8713-4b15-a71d-a10c206679c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:30.448031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-7db9a5f7-8713-4b15-a71d-a10c206679c9 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:30.450941 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7db9a5f7-8713-4b15-a71d-a10c206679c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:30.557443 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1129/2268] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:37:30.568186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bf599-c6be-4a3c-8660-16e2606acbc9 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:30.569712 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-656bf599-c6be-4a3c-8660-16e2606acbc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:30.655423 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:37:30.672943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bf599-c6be-4a3c-8660-16e2606acbc9 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:30.678229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d6793-16f0-4150-99db-ae777881e68f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:30.678976 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d6793-16f0-4150-99db-ae777881e68f service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:30.679419 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d6793-16f0-4150-99db-ae777881e68f service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:30.679838 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d6793-16f0-4150-99db-ae777881e68f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:30.692159 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d6793-16f0-4150-99db-ae777881e68f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:30.693853 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d6793-16f0-4150-99db-ae777881e68f service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:30.705082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1140/2269] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:31.039848 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9a5f7-8713-4b15-a71d-a10c206679c9 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:31.042565 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7db9a5f7-8713-4b15-a71d-a10c206679c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:31.056309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9a5f7-8713-4b15-a71d-a10c206679c9 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:31.062444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2405537-ca2f-492a-8b02-7b1292926b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:31.063421 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2405537-ca2f-492a-8b02-7b1292926b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:31.063777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2405537-ca2f-492a-8b02-7b1292926b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:31.064127 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2405537-ca2f-492a-8b02-7b1292926b0c admin admin] PATH_INFO: `/v3/projects/750141c5896f4002a4e15bbc892c6c55` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:31.065448 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2405537-ca2f-492a-8b02-7b1292926b0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=750141c5896f4002a4e15bbc892c6c55)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:31.067442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2405537-ca2f-492a-8b02-7b1292926b0c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:31.077447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:37:31.123281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b2405537-ca2f-492a-8b02-7b1292926b0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '750141c5896f4002a4e15bbc892c6c55', 'request_id': 'req-b2405537-ca2f-492a-8b02-7b1292926b0c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:31.132399 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b2405537-ca2f-492a-8b02-7b1292926b0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '750141c5896f4002a4e15bbc892c6c55', 'request_id': 'req-b2405537-ca2f-492a-8b02-7b1292926b0c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:31.139278 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b2405537-ca2f-492a-8b02-7b1292926b0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '750141c5896f4002a4e15bbc892c6c55', 'request_id': 'req-b2405537-ca2f-492a-8b02-7b1292926b0c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:31.152394 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b2405537-ca2f-492a-8b02-7b1292926b0c admin admin] The token cache is being invalidate because project 750141c5896f4002a4e15bbc892c6c55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:37:31.152696 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b2405537-ca2f-492a-8b02-7b1292926b0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:31.153489 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b2405537-ca2f-492a-8b02-7b1292926b0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:31.157205 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1130/2270] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:37:31 2022] DELETE /identity/v3/projects/750141c5896f4002a4e15bbc892c6c55 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:37:35.595920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d6793-16f0-4150-99db-ae777881e68f service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:35.605754 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d6793-16f0-4150-99db-ae777881e68f service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:35.661004 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f205667-be0b-48f7-bbe1-6b11e581df95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:35.661696 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f205667-be0b-48f7-bbe1-6b11e581df95 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:35.661985 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f205667-be0b-48f7-bbe1-6b11e581df95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:35.662269 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f205667-be0b-48f7-bbe1-6b11e581df95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:35.664083 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0f205667-be0b-48f7-bbe1-6b11e581df95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:35.726748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f205667-be0b-48f7-bbe1-6b11e581df95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:35.728175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f205667-be0b-48f7-bbe1-6b11e581df95 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:35.737972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1141/2271] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:35 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:38.815908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b8d0d-2a95-40d6-98b2-dc8f198784d4 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:38.816625 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b8d0d-2a95-40d6-98b2-dc8f198784d4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:38.816625 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b8d0d-2a95-40d6-98b2-dc8f198784d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:38.877176 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-f96b8d0d-2a95-40d6-98b2-dc8f198784d4 None None] MFA Rules not processed for user `cfc2f31f654e486d8505d80aad17d17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:38.878463 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f96b8d0d-2a95-40d6-98b2-dc8f198784d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:38.944217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1131/2272] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 17:37:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:38.971278 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f205667-be0b-48f7-bbe1-6b11e581df95 service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:38.973337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0f205667-be0b-48f7-bbe1-6b11e581df95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:39.081443 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f205667-be0b-48f7-bbe1-6b11e581df95 service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:39.087716 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ce0a5-37a7-4c0d-8ccc-1e3a63456841 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a37d1201d54afdad9b228ef4f2d787', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:39.088569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ce0a5-37a7-4c0d-8ccc-1e3a63456841 service swift] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:39.089201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ce0a5-37a7-4c0d-8ccc-1e3a63456841 service swift] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:39.089314 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ce0a5-37a7-4c0d-8ccc-1e3a63456841 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:39.097972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ce0a5-37a7-4c0d-8ccc-1e3a63456841 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:39.099555 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ce0a5-37a7-4c0d-8ccc-1e3a63456841 service swift] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:39.108921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1142/2273] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 17:37:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:37:48.609845 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b8d0d-2a95-40d6-98b2-dc8f198784d4 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:48.611350 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f96b8d0d-2a95-40d6-98b2-dc8f198784d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:48.623310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b8d0d-2a95-40d6-98b2-dc8f198784d4 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:48.626937 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2465b14-c7e5-46f9-8157-9349163b43cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:48.627591 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2465b14-c7e5-46f9-8157-9349163b43cb service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:48.627883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2465b14-c7e5-46f9-8157-9349163b43cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:48.628165 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2465b14-c7e5-46f9-8157-9349163b43cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:48.630020 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b2465b14-c7e5-46f9-8157-9349163b43cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:48.691763 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:37:48.696435 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2465b14-c7e5-46f9-8157-9349163b43cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:48.698926 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2465b14-c7e5-46f9-8157-9349163b43cb service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:48.709839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1132/2274] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:48 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:49.105935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=98 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:37:53.002344 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6146256-3511-4de8-87fe-fbf63f8d5dd5 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:53.003039 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6146256-3511-4de8-87fe-fbf63f8d5dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:53.003039 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6146256-3511-4de8-87fe-fbf63f8d5dd5 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:53.004313 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1143/2275] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 17:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:53.011888 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59555a4-06bb-49e6-ac51-982974ece00e None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:53.012370 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59555a4-06bb-49e6-ac51-982974ece00e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:53.012464 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59555a4-06bb-49e6-ac51-982974ece00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:53.050369 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d59555a4-06bb-49e6-ac51-982974ece00e None None] MFA Rules not processed for user `689a31e26c6b454a91c3fd79678b4d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:53.051600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d59555a4-06bb-49e6-ac51-982974ece00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:53.130504 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1133/2276] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 17:37:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:53.143159 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6146256-3511-4de8-87fe-fbf63f8d5dd5 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:53.144943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-e6146256-3511-4de8-87fe-fbf63f8d5dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:53.225712 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6146256-3511-4de8-87fe-fbf63f8d5dd5 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:53.229369 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1a3df-3bae-49c0-b8cf-8bf7ddd447eb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327b347d6295422c99ce7b44eab00467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:53.230019 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1a3df-3bae-49c0-b8cf-8bf7ddd447eb service glance] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:53.230340 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1a3df-3bae-49c0-b8cf-8bf7ddd447eb service glance] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:53.230623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1a3df-3bae-49c0-b8cf-8bf7ddd447eb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:53.242012 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1a3df-3bae-49c0-b8cf-8bf7ddd447eb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:53.243466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1a3df-3bae-49c0-b8cf-8bf7ddd447eb service glance] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:53.254750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1144/2277] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:53.323414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7cce7-46bb-4457-8b21-f34d98d085ee None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:53.323713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7cce7-46bb-4457-8b21-f34d98d085ee None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:53.324002 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7cce7-46bb-4457-8b21-f34d98d085ee None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:53.325457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1134/2278] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 17:37:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:37:53.330409 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab92156-ae3e-498e-8c60-ebbb959e9aea None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:53.330705 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab92156-ae3e-498e-8c60-ebbb959e9aea None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:53.330987 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab92156-ae3e-498e-8c60-ebbb959e9aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:53.350929 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-4ab92156-ae3e-498e-8c60-ebbb959e9aea None None] MFA Rules not processed for user `a303489493ce4cd2b62e876fbc8a7404`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:53.352109 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4ab92156-ae3e-498e-8c60-ebbb959e9aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:53.359100 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1145/2279] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 17:37:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:53.371832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7cce7-46bb-4457-8b21-f34d98d085ee None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:53.382833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7cce7-46bb-4457-8b21-f34d98d085ee None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:53.387044 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfedc50-b417-45ac-a163-27bee854ce24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:53.387691 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfedc50-b417-45ac-a163-27bee854ce24 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:53.387981 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfedc50-b417-45ac-a163-27bee854ce24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:53.388270 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfedc50-b417-45ac-a163-27bee854ce24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:53.395355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfedc50-b417-45ac-a163-27bee854ce24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:53.396795 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfedc50-b417-45ac-a163-27bee854ce24 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:53.405452 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1135/2280] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:53.487981 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb279a-62f1-456a-9898-dc51ef43ace1 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:53.488276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb279a-62f1-456a-9898-dc51ef43ace1 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:53.488561 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb279a-62f1-456a-9898-dc51ef43ace1 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:53.489989 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1146/2281] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 17:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:53.496506 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee405da-b3b9-4bf6-9a08-eece57b9989d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:53.496809 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee405da-b3b9-4bf6-9a08-eece57b9989d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:53.497099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee405da-b3b9-4bf6-9a08-eece57b9989d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:53.515950 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-eee405da-b3b9-4bf6-9a08-eece57b9989d None None] MFA Rules not processed for user `689a31e26c6b454a91c3fd79678b4d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:53.517201 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-eee405da-b3b9-4bf6-9a08-eece57b9989d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:53.527317 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1136/2282] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 17:37:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:37:53.537601 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb279a-62f1-456a-9898-dc51ef43ace1 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:53.539322 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-1fbb279a-62f1-456a-9898-dc51ef43ace1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:53.556804 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb279a-62f1-456a-9898-dc51ef43ace1 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:53.561757 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d46989-69c4-41f4-a01c-51d74f4667d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:53.562412 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d46989-69c4-41f4-a01c-51d74f4667d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:53.562703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d46989-69c4-41f4-a01c-51d74f4667d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:53.562989 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d46989-69c4-41f4-a01c-51d74f4667d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:53.574433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d46989-69c4-41f4-a01c-51d74f4667d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:53.575865 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d46989-69c4-41f4-a01c-51d74f4667d9 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:53.586542 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1147/2283] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:56.140629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06020a6-7a74-4c23-bf15-6bba9ed7630f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:56.141130 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06020a6-7a74-4c23-bf15-6bba9ed7630f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:56.141393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06020a6-7a74-4c23-bf15-6bba9ed7630f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:56.159320 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d06020a6-7a74-4c23-bf15-6bba9ed7630f None None] MFA Rules not processed for user `cfc2f31f654e486d8505d80aad17d17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:56.160561 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d06020a6-7a74-4c23-bf15-6bba9ed7630f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:56.167734 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1137/2284] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 17:37:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:56.179361 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d46989-69c4-41f4-a01c-51d74f4667d9 service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:56.190557 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d46989-69c4-41f4-a01c-51d74f4667d9 service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:56.195553 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e850c-534c-44a9-a60f-b7825b77e3dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a37d1201d54afdad9b228ef4f2d787', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:56.196206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e850c-534c-44a9-a60f-b7825b77e3dc service swift] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:56.196483 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e850c-534c-44a9-a60f-b7825b77e3dc service swift] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:56.197391 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e850c-534c-44a9-a60f-b7825b77e3dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:56.205719 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e850c-534c-44a9-a60f-b7825b77e3dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:56.207295 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e850c-534c-44a9-a60f-b7825b77e3dc service swift] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:56.215905 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1148/2285] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 17:37:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:38:06.154229 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=52 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:38:06.212123 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=50 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:38:21.998779 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76240b93-d18c-43e3-8a13-f3c9bbb7e598 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:21.999346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76240b93-d18c-43e3-8a13-f3c9bbb7e598 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:21.999448 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76240b93-d18c-43e3-8a13-f3c9bbb7e598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:22.018332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-76240b93-d18c-43e3-8a13-f3c9bbb7e598 None None] MFA Rules not processed for user `209ada6ba12b4bfda10045b4ee9301f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:38:22.019837 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-76240b93-d18c-43e3-8a13-f3c9bbb7e598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:38:22.078387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1138/2286] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:38:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:38:22.907106 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc951d-124b-46e7-a6e0-8c2e3618dc8c None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:22.907645 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc951d-124b-46e7-a6e0-8c2e3618dc8c None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:22.907767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc951d-124b-46e7-a6e0-8c2e3618dc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:22.926439 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-8ecc951d-124b-46e7-a6e0-8c2e3618dc8c None None] MFA Rules not processed for user `cfc2f31f654e486d8505d80aad17d17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:38:22.927661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-8ecc951d-124b-46e7-a6e0-8c2e3618dc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:38:22.935445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1149/2287] 10.222.0.40 () {58 vars in 967 bytes} [Thu Jun 16 17:38:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:38:22.949778 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76240b93-d18c-43e3-8a13-f3c9bbb7e598 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:38:22.963217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76240b93-d18c-43e3-8a13-f3c9bbb7e598 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:38:22.969610 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf195d2-3232-4e65-af97-439adf2c6405 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a37d1201d54afdad9b228ef4f2d787', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:38:22.970495 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf195d2-3232-4e65-af97-439adf2c6405 service swift] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:22.970785 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf195d2-3232-4e65-af97-439adf2c6405 service swift] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:22.971066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf195d2-3232-4e65-af97-439adf2c6405 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:22.980692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf195d2-3232-4e65-af97-439adf2c6405 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:38:22.982399 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf195d2-3232-4e65-af97-439adf2c6405 service swift] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:38:22.990805 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1139/2288] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 17:38:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:38:32.920970 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:38:32.987287 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=31 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:38:41.180984 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897050fa-dbd9-4ca6-9827-ffd910291d11 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:41.181576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897050fa-dbd9-4ca6-9827-ffd910291d11 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:41.181576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897050fa-dbd9-4ca6-9827-ffd910291d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:41.199526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-897050fa-dbd9-4ca6-9827-ffd910291d11 None None] MFA Rules not processed for user `a5c37b79bf3d40f4a6182cc296585c18`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:38:41.200750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-897050fa-dbd9-4ca6-9827-ffd910291d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:38:41.256740 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1150/2289] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:38:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:38:51.250150 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=25 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:39:10.776148 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf195d2-3232-4e65-af97-439adf2c6405 service swift] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:10.785712 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf195d2-3232-4e65-af97-439adf2c6405 service swift] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:10.790030 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57f46c-9a95-4d71-82a1-7d0652a09ba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:10.790684 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57f46c-9a95-4d71-82a1-7d0652a09ba8 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:10.790988 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57f46c-9a95-4d71-82a1-7d0652a09ba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:10.791257 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57f46c-9a95-4d71-82a1-7d0652a09ba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:10.793108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2b57f46c-9a95-4d71-82a1-7d0652a09ba8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:10.808125 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57f46c-9a95-4d71-82a1-7d0652a09ba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:10.809608 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57f46c-9a95-4d71-82a1-7d0652a09ba8 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:10.822316 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1140/2290] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:39:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:39:15.697886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0ae8d-8a1b-452a-9306-ffeab5cabc9a None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:15.698445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0ae8d-8a1b-452a-9306-ffeab5cabc9a None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:15.698445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0ae8d-8a1b-452a-9306-ffeab5cabc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:15.717207 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-e3e0ae8d-8a1b-452a-9306-ffeab5cabc9a None None] MFA Rules not processed for user `3faab41439184fecbb3e1021eb81d555`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:39:15.718429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-e3e0ae8d-8a1b-452a-9306-ffeab5cabc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:15.782036 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1151/2291] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:39:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:39:15.822157 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57f46c-9a95-4d71-82a1-7d0652a09ba8 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:15.823916 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2b57f46c-9a95-4d71-82a1-7d0652a09ba8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:15.837614 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57f46c-9a95-4d71-82a1-7d0652a09ba8 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:15.843049 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d750111-6a70-45f6-beea-b160b498fd7e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327b347d6295422c99ce7b44eab00467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:15.844348 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d750111-6a70-45f6-beea-b160b498fd7e service glance] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:15.844877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d750111-6a70-45f6-beea-b160b498fd7e service glance] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:15.845483 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d750111-6a70-45f6-beea-b160b498fd7e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:15.848776 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7d750111-6a70-45f6-beea-b160b498fd7e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:15.867839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d750111-6a70-45f6-beea-b160b498fd7e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:15.869522 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d750111-6a70-45f6-beea-b160b498fd7e service glance] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:15.880986 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1141/2292] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:39:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:39:16.556992 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355ff9b-93d6-4cc4-bce7-bb582dc08abb None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:16.557323 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355ff9b-93d6-4cc4-bce7-bb582dc08abb None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:16.557807 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355ff9b-93d6-4cc4-bce7-bb582dc08abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:16.576380 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-0355ff9b-93d6-4cc4-bce7-bb582dc08abb None None] MFA Rules not processed for user `2fbaf6d894e64f19bf041e7be20a6242`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:39:16.577650 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0355ff9b-93d6-4cc4-bce7-bb582dc08abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:16.638718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1152/2293] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:39:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:39:25.882776 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=12 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:39:25.882776 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d750111-6a70-45f6-beea-b160b498fd7e service glance] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:25.884114 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7d750111-6a70-45f6-beea-b160b498fd7e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:25.946640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d750111-6a70-45f6-beea-b160b498fd7e service glance] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:25.950300 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2ac08-7ac6-4ae8-9425-8b28985f70cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:25.950966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2ac08-7ac6-4ae8-9425-8b28985f70cd service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:25.951252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2ac08-7ac6-4ae8-9425-8b28985f70cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:25.951534 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2ac08-7ac6-4ae8-9425-8b28985f70cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:25.953405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4aa2ac08-7ac6-4ae8-9425-8b28985f70cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:25.965815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2ac08-7ac6-4ae8-9425-8b28985f70cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:25.967308 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2ac08-7ac6-4ae8-9425-8b28985f70cd service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:25.978504 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1142/2294] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:39:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:39:26.629381 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=50 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:39:29.032318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355ff9b-93d6-4cc4-bce7-bb582dc08abb None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:29.034208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0355ff9b-93d6-4cc4-bce7-bb582dc08abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:29.053930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355ff9b-93d6-4cc4-bce7-bb582dc08abb None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:29.058007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ec70e-7eb6-40b2-b2b5-234067a35100 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:29.058799 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ec70e-7eb6-40b2-b2b5-234067a35100 service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:29.059138 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ec70e-7eb6-40b2-b2b5-234067a35100 service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:29.059490 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ec70e-7eb6-40b2-b2b5-234067a35100 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:29.061452 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-e47ec70e-7eb6-40b2-b2b5-234067a35100 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:29.074749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47ec70e-7eb6-40b2-b2b5-234067a35100 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:29.076264 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47ec70e-7eb6-40b2-b2b5-234067a35100 service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:29.088331 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1153/2295] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:39:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:39:35.973353 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=26 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:39:39.082069 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=6 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:39:42.580447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f6b49-591d-4272-8f3d-961d58433102 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:42.581011 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f6b49-591d-4272-8f3d-961d58433102 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:42.581011 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f6b49-591d-4272-8f3d-961d58433102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:42.600896 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-484f6b49-591d-4272-8f3d-961d58433102 None None] MFA Rules not processed for user `209ada6ba12b4bfda10045b4ee9301f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:39:42.603100 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-484f6b49-591d-4272-8f3d-961d58433102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:42.614241 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1143/2296] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:39:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:39:42.959557 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ec70e-7eb6-40b2-b2b5-234067a35100 service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:42.973481 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ec70e-7eb6-40b2-b2b5-234067a35100 service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:42.979227 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74891456-c6af-489d-a05e-1a7d8696ed78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:42.980096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74891456-c6af-489d-a05e-1a7d8696ed78 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:42.980822 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74891456-c6af-489d-a05e-1a7d8696ed78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:42.980822 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74891456-c6af-489d-a05e-1a7d8696ed78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:42.983223 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-74891456-c6af-489d-a05e-1a7d8696ed78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:42.996992 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74891456-c6af-489d-a05e-1a7d8696ed78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:42.999099 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74891456-c6af-489d-a05e-1a7d8696ed78 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:43.010691 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1154/2297] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:39:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:39:43.384587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f6b49-591d-4272-8f3d-961d58433102 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:43.386935 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-484f6b49-591d-4272-8f3d-961d58433102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:43.470911 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f6b49-591d-4272-8f3d-961d58433102 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:43.476268 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50eee85-81d2-49d5-8816-fb6b084808fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75f0586a8c1941448d750918183f6dc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:43.476943 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50eee85-81d2-49d5-8816-fb6b084808fa service placement] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:43.477266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50eee85-81d2-49d5-8816-fb6b084808fa service placement] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:43.477552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50eee85-81d2-49d5-8816-fb6b084808fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:43.479574 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b50eee85-81d2-49d5-8816-fb6b084808fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:43.492760 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50eee85-81d2-49d5-8816-fb6b084808fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:43.494405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50eee85-81d2-49d5-8816-fb6b084808fa service placement] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:43.504715 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1144/2298] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:39:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:39:46.179997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74891456-c6af-489d-a05e-1a7d8696ed78 service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:46.190329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74891456-c6af-489d-a05e-1a7d8696ed78 service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:46.196262 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55072207-84f2-41dc-b94a-0fbf1942b995 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:46.197017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55072207-84f2-41dc-b94a-0fbf1942b995 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:46.197407 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55072207-84f2-41dc-b94a-0fbf1942b995 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:46.197796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55072207-84f2-41dc-b94a-0fbf1942b995 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:46.202025 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-55072207-84f2-41dc-b94a-0fbf1942b995 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:46.226179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55072207-84f2-41dc-b94a-0fbf1942b995 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:46.229092 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55072207-84f2-41dc-b94a-0fbf1942b995 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:46.250222 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1155/2299] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:39:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:39:47.281265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50eee85-81d2-49d5-8816-fb6b084808fa service placement] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:47.283133 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b50eee85-81d2-49d5-8816-fb6b084808fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:47.321520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50eee85-81d2-49d5-8816-fb6b084808fa service placement] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:47.339660 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d266281-c7fa-476e-8a39-9c2c60e3f0f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:47.344187 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d266281-c7fa-476e-8a39-9c2c60e3f0f1 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:47.344681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d266281-c7fa-476e-8a39-9c2c60e3f0f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:47.345201 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d266281-c7fa-476e-8a39-9c2c60e3f0f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:47.351704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8d266281-c7fa-476e-8a39-9c2c60e3f0f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:47.370054 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d266281-c7fa-476e-8a39-9c2c60e3f0f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:47.372797 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d266281-c7fa-476e-8a39-9c2c60e3f0f1 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:47.389439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1145/2300] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:39:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:39:51.429355 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55072207-84f2-41dc-b94a-0fbf1942b995 service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:51.432404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-55072207-84f2-41dc-b94a-0fbf1942b995 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:51.449607 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55072207-84f2-41dc-b94a-0fbf1942b995 service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:51.455207 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885f72c-1821-4218-aa0c-20a98ebe3bab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:51.455876 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885f72c-1821-4218-aa0c-20a98ebe3bab service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:51.456164 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885f72c-1821-4218-aa0c-20a98ebe3bab service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:51.456445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885f72c-1821-4218-aa0c-20a98ebe3bab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:51.458455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b885f72c-1821-4218-aa0c-20a98ebe3bab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:51.471406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885f72c-1821-4218-aa0c-20a98ebe3bab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:51.473190 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885f72c-1821-4218-aa0c-20a98ebe3bab service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:51.485090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1156/2301] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:39:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:39:53.455677 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d266281-c7fa-476e-8a39-9c2c60e3f0f1 service nova] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:53.467609 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d266281-c7fa-476e-8a39-9c2c60e3f0f1 service nova] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:53.472234 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b6676-cb34-40ca-a892-92acedcce0c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:53.472900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b6676-cb34-40ca-a892-92acedcce0c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:53.473215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b6676-cb34-40ca-a892-92acedcce0c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:53.473499 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b6676-cb34-40ca-a892-92acedcce0c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:53.476869 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e53b6676-cb34-40ca-a892-92acedcce0c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:53.577092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b6676-cb34-40ca-a892-92acedcce0c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:53.579017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b6676-cb34-40ca-a892-92acedcce0c3 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:53.593523 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1146/2302] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:39:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:39:58.126802 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885f72c-1821-4218-aa0c-20a98ebe3bab service nova] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:58.131876 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b885f72c-1821-4218-aa0c-20a98ebe3bab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:58.171515 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885f72c-1821-4218-aa0c-20a98ebe3bab service nova] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:58.187176 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf424562-fd1b-4128-a196-90584a20310f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:58.187604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf424562-fd1b-4128-a196-90584a20310f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:58.188164 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf424562-fd1b-4128-a196-90584a20310f admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:58.188628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf424562-fd1b-4128-a196-90584a20310f admin admin] PATH_INFO: `/v3/users/a5c37b79bf3d40f4a6182cc296585c18` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:58.191256 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf424562-fd1b-4128-a196-90584a20310f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5c37b79bf3d40f4a6182cc296585c18)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:58.195234 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf424562-fd1b-4128-a196-90584a20310f admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:58.261984 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-cf424562-fd1b-4128-a196-90584a20310f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5c37b79bf3d40f4a6182cc296585c18'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:39:58.274124 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-cf424562-fd1b-4128-a196-90584a20310f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5c37b79bf3d40f4a6182cc296585c18'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:39:58.274835 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-cf424562-fd1b-4128-a196-90584a20310f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5c37b79bf3d40f4a6182cc296585c18'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:39:58.284570 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-cf424562-fd1b-4128-a196-90584a20310f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5c37b79bf3d40f4a6182cc296585c18'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:39:58.311067 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-cf424562-fd1b-4128-a196-90584a20310f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5c37b79bf3d40f4a6182cc296585c18'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:39:58.321216 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1157/2303] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:39:58 2022] DELETE /identity/v3/users/a5c37b79bf3d40f4a6182cc296585c18 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:39:58.594565 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b6676-cb34-40ca-a892-92acedcce0c3 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:58.611376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b6676-cb34-40ca-a892-92acedcce0c3 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:58.787699 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fa4e9-422b-4581-a919-ae2790bbd022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:58.789057 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fa4e9-422b-4581-a919-ae2790bbd022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:58.789812 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fa4e9-422b-4581-a919-ae2790bbd022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:58.790305 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fa4e9-422b-4581-a919-ae2790bbd022 admin admin] PATH_INFO: `/v3/projects/d1c1d13142434356ac1210f14dad847b` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:58.794103 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fa4e9-422b-4581-a919-ae2790bbd022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1c1d13142434356ac1210f14dad847b)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:58.796071 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fa4e9-422b-4581-a919-ae2790bbd022 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:58.805294 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:39:58.869482 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-121fa4e9-422b-4581-a919-ae2790bbd022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1c1d13142434356ac1210f14dad847b', 'request_id': 'req-121fa4e9-422b-4581-a919-ae2790bbd022'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:39:58.879843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-121fa4e9-422b-4581-a919-ae2790bbd022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1c1d13142434356ac1210f14dad847b', 'request_id': 'req-121fa4e9-422b-4581-a919-ae2790bbd022'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:39:58.891230 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-121fa4e9-422b-4581-a919-ae2790bbd022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1c1d13142434356ac1210f14dad847b', 'request_id': 'req-121fa4e9-422b-4581-a919-ae2790bbd022'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:39:58.909537 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-121fa4e9-422b-4581-a919-ae2790bbd022 admin admin] The token cache is being invalidate because project d1c1d13142434356ac1210f14dad847b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:39:58.909864 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-121fa4e9-422b-4581-a919-ae2790bbd022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:39:58.910670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-121fa4e9-422b-4581-a919-ae2790bbd022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:39:58.914794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1147/2304] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:39:58 2022] DELETE /identity/v3/projects/d1c1d13142434356ac1210f14dad847b => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:39:58.920552 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abaf283-6a0c-4a61-affe-8c2f5feda057 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:58.920552 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abaf283-6a0c-4a61-affe-8c2f5feda057 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:58.920552 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abaf283-6a0c-4a61-affe-8c2f5feda057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:58.940428 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-6abaf283-6a0c-4a61-affe-8c2f5feda057 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:39:58.945033 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-6abaf283-6a0c-4a61-affe-8c2f5feda057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:59.029842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1158/2305] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:39:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:39:59.038479 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d3ec9-936a-4bf8-8762-e5290ec2f80e None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:59.038766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d3ec9-936a-4bf8-8762-e5290ec2f80e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:59.039092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d3ec9-936a-4bf8-8762-e5290ec2f80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:59.070434 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-6a9d3ec9-936a-4bf8-8762-e5290ec2f80e None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:39:59.071697 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6a9d3ec9-936a-4bf8-8762-e5290ec2f80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:59.084412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1148/2306] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:39:59.088869 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abaf283-6a0c-4a61-affe-8c2f5feda057 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:59.100163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abaf283-6a0c-4a61-affe-8c2f5feda057 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:59.105666 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab3f700-955a-4a05-b0ff-31d70ba9f37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:59.106373 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3f700-955a-4a05-b0ff-31d70ba9f37d admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:59.106674 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3f700-955a-4a05-b0ff-31d70ba9f37d admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:59.106962 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3f700-955a-4a05-b0ff-31d70ba9f37d admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:59.107715 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab3f700-955a-4a05-b0ff-31d70ba9f37d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:39:59.108037 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab3f700-955a-4a05-b0ff-31d70ba9f37d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:59.109710 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab3f700-955a-4a05-b0ff-31d70ba9f37d admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:59.132479 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1159/2307] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:39:59.134186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d3ec9-936a-4bf8-8762-e5290ec2f80e None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:59.144083 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d3ec9-936a-4bf8-8762-e5290ec2f80e None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:59.151128 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b3766-3dc2-493a-a084-60b225a4781a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:59.151814 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b3766-3dc2-493a-a084-60b225a4781a admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:59.152100 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b3766-3dc2-493a-a084-60b225a4781a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:59.152379 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b3766-3dc2-493a-a084-60b225a4781a admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:59.153108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b3766-3dc2-493a-a084-60b225a4781a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:59.155087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b3766-3dc2-493a-a084-60b225a4781a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:59.199462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1149/2308] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:39:59 2022] POST /identity/v3/projects => generated 363 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:39:59.200822 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab3f700-955a-4a05-b0ff-31d70ba9f37d admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:59.213542 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab3f700-955a-4a05-b0ff-31d70ba9f37d admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:59.238210 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:39:59.310306 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c9e14-b95b-4944-9ece-e953dacd055f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:59.311456 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c9e14-b95b-4944-9ece-e953dacd055f admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:59.311898 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c9e14-b95b-4944-9ece-e953dacd055f admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:59.312323 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c9e14-b95b-4944-9ece-e953dacd055f admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:59.313212 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c9e14-b95b-4944-9ece-e953dacd055f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:59.315537 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c9e14-b95b-4944-9ece-e953dacd055f admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:59.355192 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1160/2309] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:39:59 2022] POST /identity/v3/users => generated 282 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:39:59.356169 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b3766-3dc2-493a-a084-60b225a4781a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:59.369555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b3766-3dc2-493a-a084-60b225a4781a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:59.379361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b38d69-00f3-41f3-bf09-dc08484edae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:59.380063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b38d69-00f3-41f3-bf09-dc08484edae8 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:59.381032 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b38d69-00f3-41f3-bf09-dc08484edae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:59.381429 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b38d69-00f3-41f3-bf09-dc08484edae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:59.382187 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b38d69-00f3-41f3-bf09-dc08484edae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:59.383834 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b38d69-00f3-41f3-bf09-dc08484edae8 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:59.401410 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1150/2310] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:39:59.403251 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c9e14-b95b-4944-9ece-e953dacd055f admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:59.416173 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c9e14-b95b-4944-9ece-e953dacd055f admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:59.429551 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b845e3-ec88-4724-88d9-f838f1fdca88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:59.430374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b845e3-ec88-4724-88d9-f838f1fdca88 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:59.430844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b845e3-ec88-4724-88d9-f838f1fdca88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:59.431280 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b845e3-ec88-4724-88d9-f838f1fdca88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:59.432200 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b845e3-ec88-4724-88d9-f838f1fdca88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:59.434163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b845e3-ec88-4724-88d9-f838f1fdca88 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:59.447876 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1161/2311] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:39:59.452272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b38d69-00f3-41f3-bf09-dc08484edae8 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:59.464635 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b38d69-00f3-41f3-bf09-dc08484edae8 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:59.476187 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f140e-afb7-4048-8063-d0cf022647fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:59.477121 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f140e-afb7-4048-8063-d0cf022647fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:59.477589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f140e-afb7-4048-8063-d0cf022647fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:59.477990 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f140e-afb7-4048-8063-d0cf022647fd admin admin] PATH_INFO: `/v3/projects/345d4ec006e2462a9407bdfd06c1f872/users/5768f344fa924f5dbc6651e22461c01a/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:59.514625 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f140e-afb7-4048-8063-d0cf022647fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=345d4ec006e2462a9407bdfd06c1f872, user_id=5768f344fa924f5dbc6651e22461c01a, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:59.516809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216674c6-87b0-4342-9cd7-fc8f001c700c None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:59.517130 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216674c6-87b0-4342-9cd7-fc8f001c700c None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:59.517447 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216674c6-87b0-4342-9cd7-fc8f001c700c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:59.517952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f140e-afb7-4048-8063-d0cf022647fd admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:59.541242 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1151/2312] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:39:59 2022] PUT /identity/v3/projects/345d4ec006e2462a9407bdfd06c1f872/users/5768f344fa924f5dbc6651e22461c01a/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:39:59.541554 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-216674c6-87b0-4342-9cd7-fc8f001c700c None None] MFA Rules not processed for user `faf0a0102da24cedaedd0c67273e275b`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:39:59.543123 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-216674c6-87b0-4342-9cd7-fc8f001c700c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:59.548207 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513523d6-7295-455b-84ca-5c56d9945839 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:59.548835 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513523d6-7295-455b-84ca-5c56d9945839 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:59.549348 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513523d6-7295-455b-84ca-5c56d9945839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:59.599972 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-513523d6-7295-455b-84ca-5c56d9945839 None None] MFA Rules not processed for user `5768f344fa924f5dbc6651e22461c01a`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:39:59.602652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-513523d6-7295-455b-84ca-5c56d9945839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:59.620897 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1162/2313] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:39:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:39:59.697569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1152/2314] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:39:59 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:39:59.702037 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216674c6-87b0-4342-9cd7-fc8f001c700c None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:59.781023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216674c6-87b0-4342-9cd7-fc8f001c700c None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:59.785817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d76c8-f764-4d98-a69b-fb3f844a50ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:59.787246 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d76c8-f764-4d98-a69b-fb3f844a50ae admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:59.787679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d76c8-f764-4d98-a69b-fb3f844a50ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:59.787967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d76c8-f764-4d98-a69b-fb3f844a50ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:59.788680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d76c8-f764-4d98-a69b-fb3f844a50ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:59.790643 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d76c8-f764-4d98-a69b-fb3f844a50ae admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:59.838307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1163/2315] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:39:59 2022] POST /identity/v3/projects => generated 365 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:39:59.839920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513523d6-7295-455b-84ca-5c56d9945839 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:59.852955 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513523d6-7295-455b-84ca-5c56d9945839 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:59.874112 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:39:59.925604 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d182f-cab9-4d19-b3ac-b125b8a17642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:59.926297 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d182f-cab9-4d19-b3ac-b125b8a17642 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:59.926586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d182f-cab9-4d19-b3ac-b125b8a17642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:59.926866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d182f-cab9-4d19-b3ac-b125b8a17642 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:59.927914 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d182f-cab9-4d19-b3ac-b125b8a17642 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:59.930206 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d182f-cab9-4d19-b3ac-b125b8a17642 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:59.953588 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1153/2316] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:39:59 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:39:59.957536 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d76c8-f764-4d98-a69b-fb3f844a50ae admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:59.974967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d76c8-f764-4d98-a69b-fb3f844a50ae admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:59.982612 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cc71c6-2f62-43f0-b225-c9fae4f239c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:59.983574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc71c6-2f62-43f0-b225-c9fae4f239c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:59.984522 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc71c6-2f62-43f0-b225-c9fae4f239c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:59.984883 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc71c6-2f62-43f0-b225-c9fae4f239c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:59.986309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cc71c6-2f62-43f0-b225-c9fae4f239c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:59.988152 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cc71c6-2f62-43f0-b225-c9fae4f239c7 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:00.001054 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1164/2317] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:40:00.006556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d182f-cab9-4d19-b3ac-b125b8a17642 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:00.024715 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d182f-cab9-4d19-b3ac-b125b8a17642 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:00.035764 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093fdb27-52e4-4138-851e-2a4130e118e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:00.037676 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093fdb27-52e4-4138-851e-2a4130e118e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:00.037872 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093fdb27-52e4-4138-851e-2a4130e118e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:00.038172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093fdb27-52e4-4138-851e-2a4130e118e7 admin admin] PATH_INFO: `/v3/projects/2699a0d8756c429da2255b443ad991bc/users/00d326f11e1d499ca7717b22d552cf7d/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:00.065331 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093fdb27-52e4-4138-851e-2a4130e118e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2699a0d8756c429da2255b443ad991bc, user_id=00d326f11e1d499ca7717b22d552cf7d, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:00.072179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093fdb27-52e4-4138-851e-2a4130e118e7 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:00.107481 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1154/2318] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:40:00 2022] PUT /identity/v3/projects/2699a0d8756c429da2255b443ad991bc/users/00d326f11e1d499ca7717b22d552cf7d/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:00.107828 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cc71c6-2f62-43f0-b225-c9fae4f239c7 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:00.133738 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cc71c6-2f62-43f0-b225-c9fae4f239c7 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:00.160058 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093fdb27-52e4-4138-851e-2a4130e118e7 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:00.164049 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-093fdb27-52e4-4138-851e-2a4130e118e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:00.173072 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:00.254991 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1389b-81e6-400a-81de-dd1ed3038e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:00.258046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1389b-81e6-400a-81de-dd1ed3038e20 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:00.258649 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1389b-81e6-400a-81de-dd1ed3038e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:00.259087 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1389b-81e6-400a-81de-dd1ed3038e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:00.262338 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093fdb27-52e4-4138-851e-2a4130e118e7 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:00.262547 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1389b-81e6-400a-81de-dd1ed3038e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:00.264385 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1389b-81e6-400a-81de-dd1ed3038e20 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:00.274820 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20d267-b65a-4624-8035-fb62a04cac80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:00.275504 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20d267-b65a-4624-8035-fb62a04cac80 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:00.275797 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20d267-b65a-4624-8035-fb62a04cac80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:00.276137 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20d267-b65a-4624-8035-fb62a04cac80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:00.278574 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6a20d267-b65a-4624-8035-fb62a04cac80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:00.280809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1165/2319] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:40:00.285930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1389b-81e6-400a-81de-dd1ed3038e20 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:00.304941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1389b-81e6-400a-81de-dd1ed3038e20 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:00.310247 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20d267-b65a-4624-8035-fb62a04cac80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:00.311738 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20d267-b65a-4624-8035-fb62a04cac80 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:00.319738 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240fde5-33cd-498b-b366-251a15eb86d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:00.321318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240fde5-33cd-498b-b366-251a15eb86d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:00.322068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240fde5-33cd-498b-b366-251a15eb86d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:00.322361 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240fde5-33cd-498b-b366-251a15eb86d2 admin admin] PATH_INFO: `/v3/projects/2699a0d8756c429da2255b443ad991bc/users/00d326f11e1d499ca7717b22d552cf7d/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:00.325749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240fde5-33cd-498b-b366-251a15eb86d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2699a0d8756c429da2255b443ad991bc, user_id=00d326f11e1d499ca7717b22d552cf7d, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:00.329939 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240fde5-33cd-498b-b366-251a15eb86d2 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:00.338211 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1155/2320] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:40:00.346876 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1166/2321] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:40:00 2022] PUT /identity/v3/projects/2699a0d8756c429da2255b443ad991bc/users/00d326f11e1d499ca7717b22d552cf7d/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:00.362969 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31183d84-6d4d-4a6d-a29e-136d4439e2e5 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:00.363393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31183d84-6d4d-4a6d-a29e-136d4439e2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:00.363601 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31183d84-6d4d-4a6d-a29e-136d4439e2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:00.406134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-31183d84-6d4d-4a6d-a29e-136d4439e2e5 None None] MFA Rules not processed for user `00d326f11e1d499ca7717b22d552cf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:00.407373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-31183d84-6d4d-4a6d-a29e-136d4439e2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:00.533869 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1156/2322] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:00 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:00.550160 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240fde5-33cd-498b-b366-251a15eb86d2 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:00.554392 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-1240fde5-33cd-498b-b366-251a15eb86d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:00.651582 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240fde5-33cd-498b-b366-251a15eb86d2 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:00.657679 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631ff76-033a-4feb-baa3-3a8b313536aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:00.657801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631ff76-033a-4feb-baa3-3a8b313536aa service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:00.658144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631ff76-033a-4feb-baa3-3a8b313536aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:00.658275 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631ff76-033a-4feb-baa3-3a8b313536aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:00.686408 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631ff76-033a-4feb-baa3-3a8b313536aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:00.688538 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631ff76-033a-4feb-baa3-3a8b313536aa service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:00.703413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1167/2323] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:00 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:40:00.825716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31183d84-6d4d-4a6d-a29e-136d4439e2e5 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:00.843186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31183d84-6d4d-4a6d-a29e-136d4439e2e5 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:00.853552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25556d9-7f48-4be2-9ea1-3507fdddb899 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:00.854389 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25556d9-7f48-4be2-9ea1-3507fdddb899 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:00.854750 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25556d9-7f48-4be2-9ea1-3507fdddb899 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:00.855110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25556d9-7f48-4be2-9ea1-3507fdddb899 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:00.870352 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:00.959948 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25556d9-7f48-4be2-9ea1-3507fdddb899 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:00.961724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25556d9-7f48-4be2-9ea1-3507fdddb899 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:00.972547 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1157/2324] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:00 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:40:10.696921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:10.771524 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631ff76-033a-4feb-baa3-3a8b313536aa service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:10.839601 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631ff76-033a-4feb-baa3-3a8b313536aa service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:10.844347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6a928-f056-484f-85d7-ac23949a5bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:10.845313 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6a928-f056-484f-85d7-ac23949a5bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:10.845598 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6a928-f056-484f-85d7-ac23949a5bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:10.845875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6a928-f056-484f-85d7-ac23949a5bd1 admin admin] PATH_INFO: `/v3/users/5768f344fa924f5dbc6651e22461c01a` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:10.847080 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6a928-f056-484f-85d7-ac23949a5bd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5768f344fa924f5dbc6651e22461c01a)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:10.849118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6a928-f056-484f-85d7-ac23949a5bd1 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:10.899088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-83a6a928-f056-484f-85d7-ac23949a5bd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5768f344fa924f5dbc6651e22461c01a'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:10.906686 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-83a6a928-f056-484f-85d7-ac23949a5bd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5768f344fa924f5dbc6651e22461c01a'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:10.907309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-83a6a928-f056-484f-85d7-ac23949a5bd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5768f344fa924f5dbc6651e22461c01a'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:10.915158 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-83a6a928-f056-484f-85d7-ac23949a5bd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5768f344fa924f5dbc6651e22461c01a'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:10.927920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-83a6a928-f056-484f-85d7-ac23949a5bd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5768f344fa924f5dbc6651e22461c01a'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:10.934084 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1168/2325] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:40:10 2022] DELETE /identity/v3/users/5768f344fa924f5dbc6651e22461c01a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:10.938646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8d704-8c14-4faa-9caa-6a53fbf808ff None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:10.939141 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8d704-8c14-4faa-9caa-6a53fbf808ff None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:10.939251 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8d704-8c14-4faa-9caa-6a53fbf808ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:10.956546 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-61a8d704-8c14-4faa-9caa-6a53fbf808ff None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:10.958404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-61a8d704-8c14-4faa-9caa-6a53fbf808ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:11.028179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1158/2326] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:11.038459 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6a928-f056-484f-85d7-ac23949a5bd1 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:11.039943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-83a6a928-f056-484f-85d7-ac23949a5bd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:11.100775 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6a928-f056-484f-85d7-ac23949a5bd1 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:11.104297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e86db5-2ab1-4832-b94d-c742db70ae3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:11.104967 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e86db5-2ab1-4832-b94d-c742db70ae3d service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:11.105420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e86db5-2ab1-4832-b94d-c742db70ae3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:11.105715 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e86db5-2ab1-4832-b94d-c742db70ae3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:11.116400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e86db5-2ab1-4832-b94d-c742db70ae3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:11.117891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e86db5-2ab1-4832-b94d-c742db70ae3d service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:11.128053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1169/2327] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:40:11.481597 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8d704-8c14-4faa-9caa-6a53fbf808ff None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:11.483150 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-61a8d704-8c14-4faa-9caa-6a53fbf808ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:11.497077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8d704-8c14-4faa-9caa-6a53fbf808ff None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:11.501976 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae409a5d-d3f9-43b6-9145-6a866f8f7893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:11.502856 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae409a5d-d3f9-43b6-9145-6a866f8f7893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:11.503150 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae409a5d-d3f9-43b6-9145-6a866f8f7893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:11.503400 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae409a5d-d3f9-43b6-9145-6a866f8f7893 admin admin] PATH_INFO: `/v3/projects/345d4ec006e2462a9407bdfd06c1f872` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:11.505285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae409a5d-d3f9-43b6-9145-6a866f8f7893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=345d4ec006e2462a9407bdfd06c1f872)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:11.507266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae409a5d-d3f9-43b6-9145-6a866f8f7893 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:11.558420 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ae409a5d-d3f9-43b6-9145-6a866f8f7893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '345d4ec006e2462a9407bdfd06c1f872', 'request_id': 'req-ae409a5d-d3f9-43b6-9145-6a866f8f7893'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:11.565630 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ae409a5d-d3f9-43b6-9145-6a866f8f7893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '345d4ec006e2462a9407bdfd06c1f872', 'request_id': 'req-ae409a5d-d3f9-43b6-9145-6a866f8f7893'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:11.575834 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ae409a5d-d3f9-43b6-9145-6a866f8f7893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '345d4ec006e2462a9407bdfd06c1f872', 'request_id': 'req-ae409a5d-d3f9-43b6-9145-6a866f8f7893'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:11.590093 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ae409a5d-d3f9-43b6-9145-6a866f8f7893 admin admin] The token cache is being invalidate because project 345d4ec006e2462a9407bdfd06c1f872 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:40:11.590415 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ae409a5d-d3f9-43b6-9145-6a866f8f7893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:11.591084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ae409a5d-d3f9-43b6-9145-6a866f8f7893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:11.592647 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1159/2328] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:40:11 2022] DELETE /identity/v3/projects/345d4ec006e2462a9407bdfd06c1f872 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:11.596663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e86db5-2ab1-4832-b94d-c742db70ae3d service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:11.611443 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e86db5-2ab1-4832-b94d-c742db70ae3d service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:11.632228 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:11.678172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da5fbf-d5b1-41cc-a431-3fa2c4e3cc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:11.679481 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da5fbf-d5b1-41cc-a431-3fa2c4e3cc14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:11.679982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da5fbf-d5b1-41cc-a431-3fa2c4e3cc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:11.680335 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da5fbf-d5b1-41cc-a431-3fa2c4e3cc14 admin admin] PATH_INFO: `/v3/users/00d326f11e1d499ca7717b22d552cf7d` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:11.682350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4da5fbf-d5b1-41cc-a431-3fa2c4e3cc14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00d326f11e1d499ca7717b22d552cf7d)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:11.686096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4da5fbf-d5b1-41cc-a431-3fa2c4e3cc14 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:11.749788 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b4da5fbf-d5b1-41cc-a431-3fa2c4e3cc14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00d326f11e1d499ca7717b22d552cf7d'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:11.758856 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b4da5fbf-d5b1-41cc-a431-3fa2c4e3cc14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00d326f11e1d499ca7717b22d552cf7d'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:11.759764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b4da5fbf-d5b1-41cc-a431-3fa2c4e3cc14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00d326f11e1d499ca7717b22d552cf7d'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:11.771856 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b4da5fbf-d5b1-41cc-a431-3fa2c4e3cc14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00d326f11e1d499ca7717b22d552cf7d'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:11.790651 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b4da5fbf-d5b1-41cc-a431-3fa2c4e3cc14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00d326f11e1d499ca7717b22d552cf7d'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:11.801281 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1170/2329] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:40:11 2022] DELETE /identity/v3/users/00d326f11e1d499ca7717b22d552cf7d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:12.250565 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae409a5d-d3f9-43b6-9145-6a866f8f7893 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:12.252040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ae409a5d-d3f9-43b6-9145-6a866f8f7893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:12.282477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae409a5d-d3f9-43b6-9145-6a866f8f7893 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:12.322771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:12.352331 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a75fe-3613-4829-b99f-e7b41a5f917e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:12.353251 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a75fe-3613-4829-b99f-e7b41a5f917e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:12.353546 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a75fe-3613-4829-b99f-e7b41a5f917e admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:12.353835 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a75fe-3613-4829-b99f-e7b41a5f917e admin admin] PATH_INFO: `/v3/projects/2699a0d8756c429da2255b443ad991bc` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:12.355087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a75fe-3613-4829-b99f-e7b41a5f917e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2699a0d8756c429da2255b443ad991bc)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:12.357031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a75fe-3613-4829-b99f-e7b41a5f917e admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:12.408843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-428a75fe-3613-4829-b99f-e7b41a5f917e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2699a0d8756c429da2255b443ad991bc', 'request_id': 'req-428a75fe-3613-4829-b99f-e7b41a5f917e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:12.418254 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-428a75fe-3613-4829-b99f-e7b41a5f917e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2699a0d8756c429da2255b443ad991bc', 'request_id': 'req-428a75fe-3613-4829-b99f-e7b41a5f917e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:12.427536 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-428a75fe-3613-4829-b99f-e7b41a5f917e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2699a0d8756c429da2255b443ad991bc', 'request_id': 'req-428a75fe-3613-4829-b99f-e7b41a5f917e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:12.443157 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-428a75fe-3613-4829-b99f-e7b41a5f917e admin admin] The token cache is being invalidate because project 2699a0d8756c429da2255b443ad991bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:40:12.443977 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-428a75fe-3613-4829-b99f-e7b41a5f917e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:12.444497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-428a75fe-3613-4829-b99f-e7b41a5f917e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:12.446379 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1160/2330] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:40:12 2022] DELETE /identity/v3/projects/2699a0d8756c429da2255b443ad991bc => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:12.451082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf01833-9737-432b-a617-f31deb1fc512 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:12.451409 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf01833-9737-432b-a617-f31deb1fc512 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:12.451731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf01833-9737-432b-a617-f31deb1fc512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:12.472970 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-ddf01833-9737-432b-a617-f31deb1fc512 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:12.474522 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-ddf01833-9737-432b-a617-f31deb1fc512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:12.545783 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1171/2331] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:12.550752 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40df4b5-3708-492f-b324-e5b5783ff289 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:12.551136 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40df4b5-3708-492f-b324-e5b5783ff289 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:12.551791 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40df4b5-3708-492f-b324-e5b5783ff289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:12.572478 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a40df4b5-3708-492f-b324-e5b5783ff289 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:12.573766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a40df4b5-3708-492f-b324-e5b5783ff289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:12.584556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1161/2332] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:12.588931 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf01833-9737-432b-a617-f31deb1fc512 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:12.600096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf01833-9737-432b-a617-f31deb1fc512 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:12.604910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f1219-845f-4f90-b6f4-ef1ddd0d087a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:12.605623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f1219-845f-4f90-b6f4-ef1ddd0d087a admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:12.605922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f1219-845f-4f90-b6f4-ef1ddd0d087a admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:12.606208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f1219-845f-4f90-b6f4-ef1ddd0d087a admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:12.606962 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f1219-845f-4f90-b6f4-ef1ddd0d087a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:40:12.607277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f1219-845f-4f90-b6f4-ef1ddd0d087a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:12.608900 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f1219-845f-4f90-b6f4-ef1ddd0d087a admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:12.623644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1172/2333] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:40:12.628246 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40df4b5-3708-492f-b324-e5b5783ff289 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:12.638353 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40df4b5-3708-492f-b324-e5b5783ff289 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:12.643062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b37316-754d-4ead-acc6-1f71dd7f9fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:12.643798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b37316-754d-4ead-acc6-1f71dd7f9fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:12.644083 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b37316-754d-4ead-acc6-1f71dd7f9fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:12.644385 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b37316-754d-4ead-acc6-1f71dd7f9fe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:12.645121 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b37316-754d-4ead-acc6-1f71dd7f9fe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:12.647338 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b37316-754d-4ead-acc6-1f71dd7f9fe6 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:12.677362 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1162/2334] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:40:12 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:12.681559 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f1219-845f-4f90-b6f4-ef1ddd0d087a admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:12.694596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f1219-845f-4f90-b6f4-ef1ddd0d087a admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:12.721963 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:12.759121 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a4b72-4a7c-4592-ba27-a7ab1063fc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:12.759806 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a4b72-4a7c-4592-ba27-a7ab1063fc70 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:12.760084 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a4b72-4a7c-4592-ba27-a7ab1063fc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:12.760368 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a4b72-4a7c-4592-ba27-a7ab1063fc70 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:12.761074 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a4b72-4a7c-4592-ba27-a7ab1063fc70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:12.763031 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a4b72-4a7c-4592-ba27-a7ab1063fc70 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:12.786587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1173/2335] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:40:12 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:12.791285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b37316-754d-4ead-acc6-1f71dd7f9fe6 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:12.802907 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b37316-754d-4ead-acc6-1f71dd7f9fe6 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:12.807592 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8760978-bec6-4bd8-9a1a-2c62b402b7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:12.808268 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8760978-bec6-4bd8-9a1a-2c62b402b7ad admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:12.808559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8760978-bec6-4bd8-9a1a-2c62b402b7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:12.808840 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8760978-bec6-4bd8-9a1a-2c62b402b7ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:12.809625 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8760978-bec6-4bd8-9a1a-2c62b402b7ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:12.811649 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8760978-bec6-4bd8-9a1a-2c62b402b7ad admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:12.823692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1163/2336] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:40:12.827733 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a4b72-4a7c-4592-ba27-a7ab1063fc70 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:12.839170 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a4b72-4a7c-4592-ba27-a7ab1063fc70 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:12.843922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118730b7-d1ea-40cb-abc9-f4a9e93856d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:12.844591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118730b7-d1ea-40cb-abc9-f4a9e93856d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:12.844885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118730b7-d1ea-40cb-abc9-f4a9e93856d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:12.845188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118730b7-d1ea-40cb-abc9-f4a9e93856d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:12.845938 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118730b7-d1ea-40cb-abc9-f4a9e93856d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:12.847543 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118730b7-d1ea-40cb-abc9-f4a9e93856d8 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:12.859246 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1174/2337] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:40:12.863884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8760978-bec6-4bd8-9a1a-2c62b402b7ad admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:12.875011 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8760978-bec6-4bd8-9a1a-2c62b402b7ad admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:12.880021 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564215c2-b2d4-46fb-8798-5f7590fb127a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:12.880940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564215c2-b2d4-46fb-8798-5f7590fb127a admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:12.881259 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564215c2-b2d4-46fb-8798-5f7590fb127a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:12.881541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564215c2-b2d4-46fb-8798-5f7590fb127a admin admin] PATH_INFO: `/v3/projects/ef4a67b504994bf1ad368043426a209a/users/8d307bba7f784aa2b40532fe314a0914/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:12.903784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564215c2-b2d4-46fb-8798-5f7590fb127a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef4a67b504994bf1ad368043426a209a, user_id=8d307bba7f784aa2b40532fe314a0914, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:12.906504 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564215c2-b2d4-46fb-8798-5f7590fb127a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:12.921620 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1164/2338] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:40:12 2022] PUT /identity/v3/projects/ef4a67b504994bf1ad368043426a209a/users/8d307bba7f784aa2b40532fe314a0914/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:12.931944 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92bba3-84c0-426f-b112-850ed71dcc75 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:12.932284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92bba3-84c0-426f-b112-850ed71dcc75 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:12.932594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92bba3-84c0-426f-b112-850ed71dcc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:12.963108 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-0f92bba3-84c0-426f-b112-850ed71dcc75 None None] MFA Rules not processed for user `8d307bba7f784aa2b40532fe314a0914`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:12.964348 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0f92bba3-84c0-426f-b112-850ed71dcc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:13.043914 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1175/2339] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:13.048478 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564215c2-b2d4-46fb-8798-5f7590fb127a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:13.123076 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564215c2-b2d4-46fb-8798-5f7590fb127a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:13.127808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374f8a0-9227-4b46-9814-99bd09687c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:13.128521 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374f8a0-9227-4b46-9814-99bd09687c5d admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:13.128793 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374f8a0-9227-4b46-9814-99bd09687c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:13.129074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374f8a0-9227-4b46-9814-99bd09687c5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:13.130467 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374f8a0-9227-4b46-9814-99bd09687c5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:13.132514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374f8a0-9227-4b46-9814-99bd09687c5d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:13.136980 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:13.166311 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1165/2340] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:40:13 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:13.167453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92bba3-84c0-426f-b112-850ed71dcc75 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:13.179132 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92bba3-84c0-426f-b112-850ed71dcc75 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:13.242343 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62a85b-640b-424b-9caf-f0ca2158aa83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:13.243037 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62a85b-640b-424b-9caf-f0ca2158aa83 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:13.243329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62a85b-640b-424b-9caf-f0ca2158aa83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:13.243614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62a85b-640b-424b-9caf-f0ca2158aa83 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:13.244367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62a85b-640b-424b-9caf-f0ca2158aa83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:13.246424 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62a85b-640b-424b-9caf-f0ca2158aa83 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:13.259789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374f8a0-9227-4b46-9814-99bd09687c5d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:13.261907 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f374f8a0-9227-4b46-9814-99bd09687c5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:13.275536 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1176/2341] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:40:13 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:13.276950 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62a85b-640b-424b-9caf-f0ca2158aa83 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:13.290961 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62a85b-640b-424b-9caf-f0ca2158aa83 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:13.299988 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ce7d9-ec22-4510-88ce-a0888fe83f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:13.300889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ce7d9-ec22-4510-88ce-a0888fe83f0a admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:13.302284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ce7d9-ec22-4510-88ce-a0888fe83f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:13.302789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ce7d9-ec22-4510-88ce-a0888fe83f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:13.303821 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ce7d9-ec22-4510-88ce-a0888fe83f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:13.306677 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ce7d9-ec22-4510-88ce-a0888fe83f0a admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:13.319167 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1177/2342] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:40:13.324070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ce7d9-ec22-4510-88ce-a0888fe83f0a admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:13.334407 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ce7d9-ec22-4510-88ce-a0888fe83f0a admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:13.339169 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6d240-a1a5-43d9-a844-139adc8342c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:13.340054 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6d240-a1a5-43d9-a844-139adc8342c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:13.340348 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6d240-a1a5-43d9-a844-139adc8342c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:13.340630 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6d240-a1a5-43d9-a844-139adc8342c2 admin admin] PATH_INFO: `/v3/projects/8b7e20f5e6d44b6ebb5c839c827a2158/users/d922ce9f07c7411798a160a1df95b61b/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:13.354461 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:13.356516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374f8a0-9227-4b46-9814-99bd09687c5d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:13.357663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6d240-a1a5-43d9-a844-139adc8342c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b7e20f5e6d44b6ebb5c839c827a2158, user_id=d922ce9f07c7411798a160a1df95b61b, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:13.361675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6d240-a1a5-43d9-a844-139adc8342c2 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:13.362580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c5f37-a74a-4074-bd75-492c2b7e85a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:13.363239 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c5f37-a74a-4074-bd75-492c2b7e85a9 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:13.363531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c5f37-a74a-4074-bd75-492c2b7e85a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:13.363818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c5f37-a74a-4074-bd75-492c2b7e85a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:13.366750 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-782c5f37-a74a-4074-bd75-492c2b7e85a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:13.380329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1178/2343] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:40:13 2022] PUT /identity/v3/projects/8b7e20f5e6d44b6ebb5c839c827a2158/users/d922ce9f07c7411798a160a1df95b61b/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:13.384515 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6d240-a1a5-43d9-a844-139adc8342c2 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:13.432830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c5f37-a74a-4074-bd75-492c2b7e85a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:13.434497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c5f37-a74a-4074-bd75-492c2b7e85a9 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:13.447100 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1166/2344] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:40:13.460700 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6d240-a1a5-43d9-a844-139adc8342c2 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:13.465616 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf70ac-8402-4b2e-9514-222fcc2b83ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:13.466303 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf70ac-8402-4b2e-9514-222fcc2b83ce admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:13.466599 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf70ac-8402-4b2e-9514-222fcc2b83ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:13.466939 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf70ac-8402-4b2e-9514-222fcc2b83ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:13.467693 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcf70ac-8402-4b2e-9514-222fcc2b83ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:13.470023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcf70ac-8402-4b2e-9514-222fcc2b83ce admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:13.481721 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1179/2345] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:40:13.486428 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c5f37-a74a-4074-bd75-492c2b7e85a9 service nova] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:13.498617 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c5f37-a74a-4074-bd75-492c2b7e85a9 service nova] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:13.504354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a0b1b-324d-42f7-a395-c5c6481baddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:13.505951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a0b1b-324d-42f7-a395-c5c6481baddd admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:13.506243 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a0b1b-324d-42f7-a395-c5c6481baddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:13.506530 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a0b1b-324d-42f7-a395-c5c6481baddd admin admin] PATH_INFO: `/v3/projects/8b7e20f5e6d44b6ebb5c839c827a2158/users/d922ce9f07c7411798a160a1df95b61b/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:13.508575 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a0b1b-324d-42f7-a395-c5c6481baddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b7e20f5e6d44b6ebb5c839c827a2158, user_id=d922ce9f07c7411798a160a1df95b61b, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:13.511321 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a0b1b-324d-42f7-a395-c5c6481baddd admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:13.524501 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1167/2346] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:40:13 2022] PUT /identity/v3/projects/8b7e20f5e6d44b6ebb5c839c827a2158/users/d922ce9f07c7411798a160a1df95b61b/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:13.537136 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86e375-02db-4e6d-a81b-34799b4cf4aa None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:13.537628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86e375-02db-4e6d-a81b-34799b4cf4aa None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:13.538018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86e375-02db-4e6d-a81b-34799b4cf4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:13.566605 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-3f86e375-02db-4e6d-a81b-34799b4cf4aa None None] MFA Rules not processed for user `d922ce9f07c7411798a160a1df95b61b`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:13.568048 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-3f86e375-02db-4e6d-a81b-34799b4cf4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:13.650947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1180/2347] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:13.663293 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a0b1b-324d-42f7-a395-c5c6481baddd admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:13.665137 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a01a0b1b-324d-42f7-a395-c5c6481baddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:13.748673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a0b1b-324d-42f7-a395-c5c6481baddd admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:13.752956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f308c-ac35-4cb6-9677-80dee5a02e12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:13.754019 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f308c-ac35-4cb6-9677-80dee5a02e12 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:13.754306 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f308c-ac35-4cb6-9677-80dee5a02e12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:13.756147 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f308c-ac35-4cb6-9677-80dee5a02e12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:13.776097 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f308c-ac35-4cb6-9677-80dee5a02e12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:13.778531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f308c-ac35-4cb6-9677-80dee5a02e12 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:13.797622 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1168/2348] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:40:13.916346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86e375-02db-4e6d-a81b-34799b4cf4aa None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:13.986227 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86e375-02db-4e6d-a81b-34799b4cf4aa None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:13.991818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bedb7af-813a-4496-bae2-6f2566626f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:13.992713 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bedb7af-813a-4496-bae2-6f2566626f9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:13.993005 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bedb7af-813a-4496-bae2-6f2566626f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:13.993318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bedb7af-813a-4496-bae2-6f2566626f9f admin admin] PATH_INFO: `/v3/users/8d307bba7f784aa2b40532fe314a0914` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:13.994563 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bedb7af-813a-4496-bae2-6f2566626f9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d307bba7f784aa2b40532fe314a0914)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:13.996496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bedb7af-813a-4496-bae2-6f2566626f9f admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:14.011789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:14.050170 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-5bedb7af-813a-4496-bae2-6f2566626f9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d307bba7f784aa2b40532fe314a0914'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:14.057940 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-5bedb7af-813a-4496-bae2-6f2566626f9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d307bba7f784aa2b40532fe314a0914'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:14.058714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-5bedb7af-813a-4496-bae2-6f2566626f9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d307bba7f784aa2b40532fe314a0914'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:14.066207 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-5bedb7af-813a-4496-bae2-6f2566626f9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d307bba7f784aa2b40532fe314a0914'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:14.079806 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-5bedb7af-813a-4496-bae2-6f2566626f9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d307bba7f784aa2b40532fe314a0914'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:14.087984 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1181/2349] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:40:13 2022] DELETE /identity/v3/users/8d307bba7f784aa2b40532fe314a0914 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:14.092501 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d167a-cc1a-45f2-aaf7-8ffacf148421 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:14.092799 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d167a-cc1a-45f2-aaf7-8ffacf148421 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:14.093084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d167a-cc1a-45f2-aaf7-8ffacf148421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:14.112908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-990d167a-cc1a-45f2-aaf7-8ffacf148421 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:14.114751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-990d167a-cc1a-45f2-aaf7-8ffacf148421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:14.124444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:14.185798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1169/2350] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:14.196604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bedb7af-813a-4496-bae2-6f2566626f9f admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:14.198089 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-5bedb7af-813a-4496-bae2-6f2566626f9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:14.265975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bedb7af-813a-4496-bae2-6f2566626f9f admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:14.269735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f39096-39a0-430d-8514-928bd32829a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:14.270449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f39096-39a0-430d-8514-928bd32829a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:14.270742 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f39096-39a0-430d-8514-928bd32829a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:14.271196 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f39096-39a0-430d-8514-928bd32829a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:14.281565 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f39096-39a0-430d-8514-928bd32829a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:14.283027 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f39096-39a0-430d-8514-928bd32829a2 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:14.293746 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1182/2351] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:40:14.598159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d167a-cc1a-45f2-aaf7-8ffacf148421 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:14.600503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-990d167a-cc1a-45f2-aaf7-8ffacf148421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:14.616674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d167a-cc1a-45f2-aaf7-8ffacf148421 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:14.622493 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138b689-ce77-44dc-b997-94b6de599f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:14.623372 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138b689-ce77-44dc-b997-94b6de599f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:14.623659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138b689-ce77-44dc-b997-94b6de599f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:14.623939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138b689-ce77-44dc-b997-94b6de599f28 admin admin] PATH_INFO: `/v3/projects/ef4a67b504994bf1ad368043426a209a` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:14.625230 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5138b689-ce77-44dc-b997-94b6de599f28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef4a67b504994bf1ad368043426a209a)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:14.627080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5138b689-ce77-44dc-b997-94b6de599f28 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:14.681127 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5138b689-ce77-44dc-b997-94b6de599f28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef4a67b504994bf1ad368043426a209a', 'request_id': 'req-5138b689-ce77-44dc-b997-94b6de599f28'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:14.689480 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5138b689-ce77-44dc-b997-94b6de599f28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef4a67b504994bf1ad368043426a209a', 'request_id': 'req-5138b689-ce77-44dc-b997-94b6de599f28'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:14.697965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5138b689-ce77-44dc-b997-94b6de599f28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef4a67b504994bf1ad368043426a209a', 'request_id': 'req-5138b689-ce77-44dc-b997-94b6de599f28'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:14.709589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5138b689-ce77-44dc-b997-94b6de599f28 admin admin] The token cache is being invalidate because project ef4a67b504994bf1ad368043426a209a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:40:14.709901 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5138b689-ce77-44dc-b997-94b6de599f28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:14.710595 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5138b689-ce77-44dc-b997-94b6de599f28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:14.712212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1170/2352] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:40:14 2022] DELETE /identity/v3/projects/ef4a67b504994bf1ad368043426a209a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:40:14.717713 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f39096-39a0-430d-8514-928bd32829a2 service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:14.729095 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f39096-39a0-430d-8514-928bd32829a2 service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:14.795839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13027118-2e67-44ee-bd4e-c4c61bdfa258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:14.796754 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13027118-2e67-44ee-bd4e-c4c61bdfa258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:14.797069 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13027118-2e67-44ee-bd4e-c4c61bdfa258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:14.797387 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13027118-2e67-44ee-bd4e-c4c61bdfa258 admin admin] PATH_INFO: `/v3/users/d922ce9f07c7411798a160a1df95b61b` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:14.798785 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13027118-2e67-44ee-bd4e-c4c61bdfa258 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d922ce9f07c7411798a160a1df95b61b)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:14.801068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13027118-2e67-44ee-bd4e-c4c61bdfa258 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:14.877551 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:14.898155 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-13027118-2e67-44ee-bd4e-c4c61bdfa258 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd922ce9f07c7411798a160a1df95b61b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:14.913069 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-13027118-2e67-44ee-bd4e-c4c61bdfa258 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd922ce9f07c7411798a160a1df95b61b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:14.914200 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-13027118-2e67-44ee-bd4e-c4c61bdfa258 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd922ce9f07c7411798a160a1df95b61b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:14.925404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-13027118-2e67-44ee-bd4e-c4c61bdfa258 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd922ce9f07c7411798a160a1df95b61b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:14.941106 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-13027118-2e67-44ee-bd4e-c4c61bdfa258 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd922ce9f07c7411798a160a1df95b61b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:14.948623 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1183/2353] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:40:14 2022] DELETE /identity/v3/users/d922ce9f07c7411798a160a1df95b61b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:15.450029 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138b689-ce77-44dc-b997-94b6de599f28 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:15.451432 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5138b689-ce77-44dc-b997-94b6de599f28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:15.476736 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138b689-ce77-44dc-b997-94b6de599f28 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:15.591807 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c7b6f-2c13-4f43-8cc8-b937b788a159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:15.592688 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c7b6f-2c13-4f43-8cc8-b937b788a159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:15.593011 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c7b6f-2c13-4f43-8cc8-b937b788a159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:15.593366 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c7b6f-2c13-4f43-8cc8-b937b788a159 admin admin] PATH_INFO: `/v3/projects/8b7e20f5e6d44b6ebb5c839c827a2158` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:15.595168 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c7b6f-2c13-4f43-8cc8-b937b788a159 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b7e20f5e6d44b6ebb5c839c827a2158)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:15.597106 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c7b6f-2c13-4f43-8cc8-b937b788a159 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:15.661996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:15.670774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6b4c7b6f-2c13-4f43-8cc8-b937b788a159 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b7e20f5e6d44b6ebb5c839c827a2158', 'request_id': 'req-6b4c7b6f-2c13-4f43-8cc8-b937b788a159'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:15.682481 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6b4c7b6f-2c13-4f43-8cc8-b937b788a159 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b7e20f5e6d44b6ebb5c839c827a2158', 'request_id': 'req-6b4c7b6f-2c13-4f43-8cc8-b937b788a159'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:15.690891 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6b4c7b6f-2c13-4f43-8cc8-b937b788a159 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b7e20f5e6d44b6ebb5c839c827a2158', 'request_id': 'req-6b4c7b6f-2c13-4f43-8cc8-b937b788a159'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:15.703276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6b4c7b6f-2c13-4f43-8cc8-b937b788a159 admin admin] The token cache is being invalidate because project 8b7e20f5e6d44b6ebb5c839c827a2158 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:40:15.703579 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6b4c7b6f-2c13-4f43-8cc8-b937b788a159 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:15.704249 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-6b4c7b6f-2c13-4f43-8cc8-b937b788a159 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:15.705859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1171/2354] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:40:15 2022] DELETE /identity/v3/projects/8b7e20f5e6d44b6ebb5c839c827a2158 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:15.710892 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b701c15-9f62-457a-858a-b988aed4fe13 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:15.711125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b701c15-9f62-457a-858a-b988aed4fe13 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:15.711415 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b701c15-9f62-457a-858a-b988aed4fe13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:15.730962 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-7b701c15-9f62-457a-858a-b988aed4fe13 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:15.732187 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-7b701c15-9f62-457a-858a-b988aed4fe13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:15.801463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1184/2355] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:15.809567 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77efc59-2006-417c-8d5c-14f1449ff9f6 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:15.810124 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77efc59-2006-417c-8d5c-14f1449ff9f6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:15.810414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77efc59-2006-417c-8d5c-14f1449ff9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:15.830459 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-b77efc59-2006-417c-8d5c-14f1449ff9f6 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:15.831879 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b77efc59-2006-417c-8d5c-14f1449ff9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:15.842016 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1172/2356] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:15.850316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b701c15-9f62-457a-858a-b988aed4fe13 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:15.862241 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b701c15-9f62-457a-858a-b988aed4fe13 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:15.869190 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04b955-54ee-4e90-b97d-c4dde05a905e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:15.869884 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04b955-54ee-4e90-b97d-c4dde05a905e admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:15.870335 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04b955-54ee-4e90-b97d-c4dde05a905e admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:15.870753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04b955-54ee-4e90-b97d-c4dde05a905e admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:15.871519 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04b955-54ee-4e90-b97d-c4dde05a905e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:40:15.871841 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04b955-54ee-4e90-b97d-c4dde05a905e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:15.873625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04b955-54ee-4e90-b97d-c4dde05a905e admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:15.887993 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1185/2357] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:40:15.892362 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77efc59-2006-417c-8d5c-14f1449ff9f6 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:15.902586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77efc59-2006-417c-8d5c-14f1449ff9f6 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:15.907745 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572d309-4b3b-4bf4-841b-e4e4adc1afa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:15.908432 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572d309-4b3b-4bf4-841b-e4e4adc1afa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:15.908727 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572d309-4b3b-4bf4-841b-e4e4adc1afa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:15.909011 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572d309-4b3b-4bf4-841b-e4e4adc1afa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:15.909902 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572d309-4b3b-4bf4-841b-e4e4adc1afa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:15.913402 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572d309-4b3b-4bf4-841b-e4e4adc1afa1 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:15.948006 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1173/2358] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:40:15 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:15.949189 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04b955-54ee-4e90-b97d-c4dde05a905e admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:15.962324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04b955-54ee-4e90-b97d-c4dde05a905e admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:16.033421 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe87d6-fbae-4c93-9f33-7a215abfb055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:16.034289 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe87d6-fbae-4c93-9f33-7a215abfb055 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:16.034746 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe87d6-fbae-4c93-9f33-7a215abfb055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:16.035130 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe87d6-fbae-4c93-9f33-7a215abfb055 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:16.036591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe87d6-fbae-4c93-9f33-7a215abfb055 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:16.038813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe87d6-fbae-4c93-9f33-7a215abfb055 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:16.062232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1186/2359] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:40:15 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:40:16.069468 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572d309-4b3b-4bf4-841b-e4e4adc1afa1 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:16.085921 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572d309-4b3b-4bf4-841b-e4e4adc1afa1 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:16.091978 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ff8e3-28a7-4e48-8648-80085335b73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:16.092575 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ff8e3-28a7-4e48-8648-80085335b73b admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:16.092929 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ff8e3-28a7-4e48-8648-80085335b73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:16.093181 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ff8e3-28a7-4e48-8648-80085335b73b admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:16.093965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ff8e3-28a7-4e48-8648-80085335b73b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:16.095590 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ff8e3-28a7-4e48-8648-80085335b73b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:16.108208 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1174/2360] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:40:16.112862 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe87d6-fbae-4c93-9f33-7a215abfb055 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:16.124899 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe87d6-fbae-4c93-9f33-7a215abfb055 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:16.135029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a783af-d288-4bee-b3b5-3e5680832b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:16.135746 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a783af-d288-4bee-b3b5-3e5680832b34 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:16.136058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a783af-d288-4bee-b3b5-3e5680832b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:16.136343 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a783af-d288-4bee-b3b5-3e5680832b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:16.137084 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a783af-d288-4bee-b3b5-3e5680832b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:16.138799 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a783af-d288-4bee-b3b5-3e5680832b34 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:16.156598 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1187/2361] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:40:16.158698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ff8e3-28a7-4e48-8648-80085335b73b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:16.171977 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ff8e3-28a7-4e48-8648-80085335b73b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:16.177950 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d7f46-7c11-45f4-b731-dabe86869221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:16.178880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d7f46-7c11-45f4-b731-dabe86869221 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:16.179195 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d7f46-7c11-45f4-b731-dabe86869221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:16.179483 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d7f46-7c11-45f4-b731-dabe86869221 admin admin] PATH_INFO: `/v3/projects/fe217c8ef64c4c82b2f68e6319d257fa/users/47ee4653adc14acda719ffd01cb0b274/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:16.199174 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d7f46-7c11-45f4-b731-dabe86869221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe217c8ef64c4c82b2f68e6319d257fa, user_id=47ee4653adc14acda719ffd01cb0b274, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:16.203552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d7f46-7c11-45f4-b731-dabe86869221 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:16.228586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1175/2362] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:40:16 2022] PUT /identity/v3/projects/fe217c8ef64c4c82b2f68e6319d257fa/users/47ee4653adc14acda719ffd01cb0b274/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:40:16.241352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf969fb2-996d-4b5d-88f9-8ab7830edaf4 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:16.241840 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf969fb2-996d-4b5d-88f9-8ab7830edaf4 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:16.242193 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf969fb2-996d-4b5d-88f9-8ab7830edaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:16.276335 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-cf969fb2-996d-4b5d-88f9-8ab7830edaf4 None None] MFA Rules not processed for user `47ee4653adc14acda719ffd01cb0b274`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:16.277902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-cf969fb2-996d-4b5d-88f9-8ab7830edaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:16.288789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:16.359749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1188/2363] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:16.365049 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d7f46-7c11-45f4-b731-dabe86869221 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:16.438835 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d7f46-7c11-45f4-b731-dabe86869221 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:16.447832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04699a2c-05de-4b8b-aabb-264e021ee6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:16.448780 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04699a2c-05de-4b8b-aabb-264e021ee6ae admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:16.449177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04699a2c-05de-4b8b-aabb-264e021ee6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:16.449482 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04699a2c-05de-4b8b-aabb-264e021ee6ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:16.450396 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04699a2c-05de-4b8b-aabb-264e021ee6ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:16.452826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04699a2c-05de-4b8b-aabb-264e021ee6ae admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:16.495405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1176/2364] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:40:16 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:16.497119 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf969fb2-996d-4b5d-88f9-8ab7830edaf4 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:16.509728 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf969fb2-996d-4b5d-88f9-8ab7830edaf4 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:16.582937 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83ff25-1933-409c-a23f-09e9b5a7d968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:16.583791 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83ff25-1933-409c-a23f-09e9b5a7d968 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:16.584087 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83ff25-1933-409c-a23f-09e9b5a7d968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:16.584369 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83ff25-1933-409c-a23f-09e9b5a7d968 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:16.585114 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c83ff25-1933-409c-a23f-09e9b5a7d968 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:16.587125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c83ff25-1933-409c-a23f-09e9b5a7d968 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:16.609732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1189/2365] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:40:16 2022] POST /identity/v3/users => generated 298 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:16.614451 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04699a2c-05de-4b8b-aabb-264e021ee6ae admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:16.625327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04699a2c-05de-4b8b-aabb-264e021ee6ae admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:16.631200 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64341e-0310-4182-ad12-23e4ef08ad41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:16.631786 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64341e-0310-4182-ad12-23e4ef08ad41 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:16.632074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64341e-0310-4182-ad12-23e4ef08ad41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:16.632363 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64341e-0310-4182-ad12-23e4ef08ad41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:16.633112 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f64341e-0310-4182-ad12-23e4ef08ad41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:16.634917 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f64341e-0310-4182-ad12-23e4ef08ad41 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:16.647812 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1177/2366] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:40:16.651701 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83ff25-1933-409c-a23f-09e9b5a7d968 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:16.662914 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83ff25-1933-409c-a23f-09e9b5a7d968 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:16.669584 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f169ff-ecf2-42cc-81dc-e68b2e9b1a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:16.670611 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f169ff-ecf2-42cc-81dc-e68b2e9b1a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:16.670982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f169ff-ecf2-42cc-81dc-e68b2e9b1a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:16.671338 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f169ff-ecf2-42cc-81dc-e68b2e9b1a0b admin admin] PATH_INFO: `/v3/projects/c89b8fc4183a4aabae2d41005d76b785/users/f4b5850c604748fdac15b6e181506606/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:16.695126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f169ff-ecf2-42cc-81dc-e68b2e9b1a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c89b8fc4183a4aabae2d41005d76b785, user_id=f4b5850c604748fdac15b6e181506606, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:16.698950 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f169ff-ecf2-42cc-81dc-e68b2e9b1a0b admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:16.731359 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1190/2367] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:40:16 2022] PUT /identity/v3/projects/c89b8fc4183a4aabae2d41005d76b785/users/f4b5850c604748fdac15b6e181506606/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:40:16.735595 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64341e-0310-4182-ad12-23e4ef08ad41 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:16.748185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64341e-0310-4182-ad12-23e4ef08ad41 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:16.814530 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:16.847325 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d299d5-7411-4072-aa5c-d160a26e4b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:16.848166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d299d5-7411-4072-aa5c-d160a26e4b2a admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:16.848544 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d299d5-7411-4072-aa5c-d160a26e4b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:16.848544 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d299d5-7411-4072-aa5c-d160a26e4b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:16.849707 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d299d5-7411-4072-aa5c-d160a26e4b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:16.851689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d299d5-7411-4072-aa5c-d160a26e4b2a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:16.862772 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1178/2368] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:40:16.867918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f169ff-ecf2-42cc-81dc-e68b2e9b1a0b admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:16.887686 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f169ff-ecf2-42cc-81dc-e68b2e9b1a0b admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:16.892741 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf82ce2-bfc0-48d4-928e-22c592e7aff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:16.893692 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf82ce2-bfc0-48d4-928e-22c592e7aff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:16.893980 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf82ce2-bfc0-48d4-928e-22c592e7aff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:16.894254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf82ce2-bfc0-48d4-928e-22c592e7aff8 admin admin] PATH_INFO: `/v3/projects/c89b8fc4183a4aabae2d41005d76b785/users/f4b5850c604748fdac15b6e181506606/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:16.897025 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf82ce2-bfc0-48d4-928e-22c592e7aff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c89b8fc4183a4aabae2d41005d76b785, user_id=f4b5850c604748fdac15b6e181506606, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:16.899767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf82ce2-bfc0-48d4-928e-22c592e7aff8 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:16.919369 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1191/2369] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:40:16 2022] PUT /identity/v3/projects/c89b8fc4183a4aabae2d41005d76b785/users/f4b5850c604748fdac15b6e181506606/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:16.931777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fe27f-a1a7-4797-805b-5c2645dfb9d3 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:16.932346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fe27f-a1a7-4797-805b-5c2645dfb9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:16.933090 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fe27f-a1a7-4797-805b-5c2645dfb9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:16.978723 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e81fe27f-a1a7-4797-805b-5c2645dfb9d3 None None] MFA Rules not processed for user `f4b5850c604748fdac15b6e181506606`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:16.980360 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e81fe27f-a1a7-4797-805b-5c2645dfb9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:17.078441 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1179/2370] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:17.099308 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf82ce2-bfc0-48d4-928e-22c592e7aff8 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:17.101353 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-bcf82ce2-bfc0-48d4-928e-22c592e7aff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:17.199098 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf82ce2-bfc0-48d4-928e-22c592e7aff8 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:17.203147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654b016-e417-42f4-a467-1fbb35183072 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:17.203806 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654b016-e417-42f4-a467-1fbb35183072 service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:17.204100 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654b016-e417-42f4-a467-1fbb35183072 service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:17.204387 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654b016-e417-42f4-a467-1fbb35183072 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:17.220294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:17.281458 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654b016-e417-42f4-a467-1fbb35183072 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:17.283040 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654b016-e417-42f4-a467-1fbb35183072 service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:17.294252 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1192/2371] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:40:17.573835 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fe27f-a1a7-4797-805b-5c2645dfb9d3 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:17.575285 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e81fe27f-a1a7-4797-805b-5c2645dfb9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:17.672452 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fe27f-a1a7-4797-805b-5c2645dfb9d3 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:17.678967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7315d-ebd9-4891-82c6-98b0f9872477 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:17.680016 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7315d-ebd9-4891-82c6-98b0f9872477 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:17.680326 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7315d-ebd9-4891-82c6-98b0f9872477 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:17.680690 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7315d-ebd9-4891-82c6-98b0f9872477 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:17.682980 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-8be7315d-ebd9-4891-82c6-98b0f9872477 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:17.698667 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7315d-ebd9-4891-82c6-98b0f9872477 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:17.700404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7315d-ebd9-4891-82c6-98b0f9872477 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:17.715823 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1180/2372] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:40:22.006368 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654b016-e417-42f4-a467-1fbb35183072 service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:22.007957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-e654b016-e417-42f4-a467-1fbb35183072 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:22.019248 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654b016-e417-42f4-a467-1fbb35183072 service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:22.024538 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a33da-9050-42c8-80df-c086328f022b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:22.025211 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a33da-9050-42c8-80df-c086328f022b service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:22.025521 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a33da-9050-42c8-80df-c086328f022b service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:22.025799 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a33da-9050-42c8-80df-c086328f022b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:22.036397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6a33da-9050-42c8-80df-c086328f022b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:22.037909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6a33da-9050-42c8-80df-c086328f022b service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:22.048355 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1193/2373] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:40:27.348802 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7315d-ebd9-4891-82c6-98b0f9872477 service nova] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:27.398142 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:27.415903 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7315d-ebd9-4891-82c6-98b0f9872477 service nova] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:27.420455 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30919d4f-2fb2-4b8e-90ed-9ec7538d88c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:27.421369 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30919d4f-2fb2-4b8e-90ed-9ec7538d88c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:27.421660 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30919d4f-2fb2-4b8e-90ed-9ec7538d88c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:27.421943 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30919d4f-2fb2-4b8e-90ed-9ec7538d88c1 admin admin] PATH_INFO: `/v3/users/47ee4653adc14acda719ffd01cb0b274` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:27.423144 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30919d4f-2fb2-4b8e-90ed-9ec7538d88c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47ee4653adc14acda719ffd01cb0b274)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:27.425098 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30919d4f-2fb2-4b8e-90ed-9ec7538d88c1 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:27.475979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-30919d4f-2fb2-4b8e-90ed-9ec7538d88c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47ee4653adc14acda719ffd01cb0b274'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:27.488803 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-30919d4f-2fb2-4b8e-90ed-9ec7538d88c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47ee4653adc14acda719ffd01cb0b274'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:27.490094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-30919d4f-2fb2-4b8e-90ed-9ec7538d88c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47ee4653adc14acda719ffd01cb0b274'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:27.497725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-30919d4f-2fb2-4b8e-90ed-9ec7538d88c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47ee4653adc14acda719ffd01cb0b274'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:27.508230 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-30919d4f-2fb2-4b8e-90ed-9ec7538d88c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47ee4653adc14acda719ffd01cb0b274'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:27.514959 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1181/2374] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:40:27 2022] DELETE /identity/v3/users/47ee4653adc14acda719ffd01cb0b274 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:27.520163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5251c3-7b75-488c-9d3f-976afb7da17a None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:27.520670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5251c3-7b75-488c-9d3f-976afb7da17a None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:27.520768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5251c3-7b75-488c-9d3f-976afb7da17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:27.539404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-6b5251c3-7b75-488c-9d3f-976afb7da17a None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:27.540622 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-6b5251c3-7b75-488c-9d3f-976afb7da17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:27.601929 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1194/2375] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:27.611865 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30919d4f-2fb2-4b8e-90ed-9ec7538d88c1 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:27.613615 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-30919d4f-2fb2-4b8e-90ed-9ec7538d88c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:27.671435 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30919d4f-2fb2-4b8e-90ed-9ec7538d88c1 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:27.675080 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be16d0c-bb92-4768-b5c0-4da22ed52f2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:27.675730 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be16d0c-bb92-4768-b5c0-4da22ed52f2c service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:27.676016 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be16d0c-bb92-4768-b5c0-4da22ed52f2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:27.676295 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be16d0c-bb92-4768-b5c0-4da22ed52f2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:27.686247 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be16d0c-bb92-4768-b5c0-4da22ed52f2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:27.687671 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be16d0c-bb92-4768-b5c0-4da22ed52f2c service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:27.698131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1182/2376] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:40:28.066569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5251c3-7b75-488c-9d3f-976afb7da17a None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:28.068601 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-6b5251c3-7b75-488c-9d3f-976afb7da17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:28.081924 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5251c3-7b75-488c-9d3f-976afb7da17a None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:28.087201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbdcc43-f30d-4904-9cf9-5010dd80d9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:28.088079 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbdcc43-f30d-4904-9cf9-5010dd80d9b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:28.088372 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbdcc43-f30d-4904-9cf9-5010dd80d9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:28.088652 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbdcc43-f30d-4904-9cf9-5010dd80d9b0 admin admin] PATH_INFO: `/v3/projects/fe217c8ef64c4c82b2f68e6319d257fa` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:28.090084 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbdcc43-f30d-4904-9cf9-5010dd80d9b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe217c8ef64c4c82b2f68e6319d257fa)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:28.091999 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbdcc43-f30d-4904-9cf9-5010dd80d9b0 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:28.145733 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-5bbdcc43-f30d-4904-9cf9-5010dd80d9b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe217c8ef64c4c82b2f68e6319d257fa', 'request_id': 'req-5bbdcc43-f30d-4904-9cf9-5010dd80d9b0'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:28.152083 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-5bbdcc43-f30d-4904-9cf9-5010dd80d9b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe217c8ef64c4c82b2f68e6319d257fa', 'request_id': 'req-5bbdcc43-f30d-4904-9cf9-5010dd80d9b0'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:28.158402 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-5bbdcc43-f30d-4904-9cf9-5010dd80d9b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe217c8ef64c4c82b2f68e6319d257fa', 'request_id': 'req-5bbdcc43-f30d-4904-9cf9-5010dd80d9b0'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:28.172134 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-5bbdcc43-f30d-4904-9cf9-5010dd80d9b0 admin admin] The token cache is being invalidate because project fe217c8ef64c4c82b2f68e6319d257fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:40:28.172429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-5bbdcc43-f30d-4904-9cf9-5010dd80d9b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:28.173062 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-5bbdcc43-f30d-4904-9cf9-5010dd80d9b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:28.174830 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1195/2377] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:40:28 2022] DELETE /identity/v3/projects/fe217c8ef64c4c82b2f68e6319d257fa => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:28.179196 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be16d0c-bb92-4768-b5c0-4da22ed52f2c service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:28.189103 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be16d0c-bb92-4768-b5c0-4da22ed52f2c service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:28.248071 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf8c37-2973-49e4-9fdf-ab0e21ac642c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:28.248957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf8c37-2973-49e4-9fdf-ab0e21ac642c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:28.249283 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf8c37-2973-49e4-9fdf-ab0e21ac642c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:28.249568 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf8c37-2973-49e4-9fdf-ab0e21ac642c admin admin] PATH_INFO: `/v3/users/f4b5850c604748fdac15b6e181506606` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:28.250751 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf8c37-2973-49e4-9fdf-ab0e21ac642c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4b5850c604748fdac15b6e181506606)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:28.252690 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf8c37-2973-49e4-9fdf-ab0e21ac642c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:28.258127 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:28.300700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d4cf8c37-2973-49e4-9fdf-ab0e21ac642c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4b5850c604748fdac15b6e181506606'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:28.312578 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d4cf8c37-2973-49e4-9fdf-ab0e21ac642c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4b5850c604748fdac15b6e181506606'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:28.313284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d4cf8c37-2973-49e4-9fdf-ab0e21ac642c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4b5850c604748fdac15b6e181506606'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:28.320900 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d4cf8c37-2973-49e4-9fdf-ab0e21ac642c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4b5850c604748fdac15b6e181506606'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:28.328726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-d4cf8c37-2973-49e4-9fdf-ab0e21ac642c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4b5850c604748fdac15b6e181506606'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:28.335742 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1183/2378] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:40:28 2022] DELETE /identity/v3/users/f4b5850c604748fdac15b6e181506606 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:40:28.617956 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbdcc43-f30d-4904-9cf9-5010dd80d9b0 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:28.619580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-5bbdcc43-f30d-4904-9cf9-5010dd80d9b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:28.639413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbdcc43-f30d-4904-9cf9-5010dd80d9b0 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:28.666438 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:28.701413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3212f1-869c-4763-a42d-cf62c6cbb546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:28.702294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3212f1-869c-4763-a42d-cf62c6cbb546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:28.702615 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3212f1-869c-4763-a42d-cf62c6cbb546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:28.702899 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3212f1-869c-4763-a42d-cf62c6cbb546 admin admin] PATH_INFO: `/v3/projects/c89b8fc4183a4aabae2d41005d76b785` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:28.704134 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3212f1-869c-4763-a42d-cf62c6cbb546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c89b8fc4183a4aabae2d41005d76b785)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:28.706170 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3212f1-869c-4763-a42d-cf62c6cbb546 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:28.753655 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-ea3212f1-869c-4763-a42d-cf62c6cbb546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c89b8fc4183a4aabae2d41005d76b785', 'request_id': 'req-ea3212f1-869c-4763-a42d-cf62c6cbb546'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:28.760000 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-ea3212f1-869c-4763-a42d-cf62c6cbb546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c89b8fc4183a4aabae2d41005d76b785', 'request_id': 'req-ea3212f1-869c-4763-a42d-cf62c6cbb546'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:28.766182 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-ea3212f1-869c-4763-a42d-cf62c6cbb546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c89b8fc4183a4aabae2d41005d76b785', 'request_id': 'req-ea3212f1-869c-4763-a42d-cf62c6cbb546'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:28.780088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-ea3212f1-869c-4763-a42d-cf62c6cbb546 admin admin] The token cache is being invalidate because project c89b8fc4183a4aabae2d41005d76b785 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:40:28.780390 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-ea3212f1-869c-4763-a42d-cf62c6cbb546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:28.781113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-ea3212f1-869c-4763-a42d-cf62c6cbb546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:28.782660 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1196/2379] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:40:28 2022] DELETE /identity/v3/projects/c89b8fc4183a4aabae2d41005d76b785 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:28.793528 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58911c-fd76-4c1e-a831-c3f7f87498ec None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:28.794046 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58911c-fd76-4c1e-a831-c3f7f87498ec None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:28.794046 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58911c-fd76-4c1e-a831-c3f7f87498ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:28.815933 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-9c58911c-fd76-4c1e-a831-c3f7f87498ec None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:28.817202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9c58911c-fd76-4c1e-a831-c3f7f87498ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:28.883434 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1184/2380] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:28.891587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff9f5e-fd51-4be4-a5a5-7bb28396cc65 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:28.891888 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff9f5e-fd51-4be4-a5a5-7bb28396cc65 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:28.892178 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff9f5e-fd51-4be4-a5a5-7bb28396cc65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:28.913319 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-0bff9f5e-fd51-4be4-a5a5-7bb28396cc65 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:28.915000 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0bff9f5e-fd51-4be4-a5a5-7bb28396cc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:28.923239 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1197/2381] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:28.928254 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58911c-fd76-4c1e-a831-c3f7f87498ec None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:28.938317 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58911c-fd76-4c1e-a831-c3f7f87498ec None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:28.943873 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be08c66-5953-4e84-8314-f2d7b9ca27c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:28.944674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be08c66-5953-4e84-8314-f2d7b9ca27c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:28.944931 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be08c66-5953-4e84-8314-f2d7b9ca27c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:28.945264 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be08c66-5953-4e84-8314-f2d7b9ca27c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:28.946230 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be08c66-5953-4e84-8314-f2d7b9ca27c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:40:28.946758 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be08c66-5953-4e84-8314-f2d7b9ca27c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:28.948465 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be08c66-5953-4e84-8314-f2d7b9ca27c6 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:28.963343 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1185/2382] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:40:28.967780 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff9f5e-fd51-4be4-a5a5-7bb28396cc65 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:28.982002 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff9f5e-fd51-4be4-a5a5-7bb28396cc65 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:28.987030 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e91be8-9f47-4eb6-aaa7-3d8577a320e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:28.987726 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e91be8-9f47-4eb6-aaa7-3d8577a320e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:28.988020 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e91be8-9f47-4eb6-aaa7-3d8577a320e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:28.988303 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e91be8-9f47-4eb6-aaa7-3d8577a320e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:28.989018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e91be8-9f47-4eb6-aaa7-3d8577a320e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:28.991184 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e91be8-9f47-4eb6-aaa7-3d8577a320e2 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:29.033517 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1198/2383] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:40:28 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:29.034952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be08c66-5953-4e84-8314-f2d7b9ca27c6 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:29.046310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be08c66-5953-4e84-8314-f2d7b9ca27c6 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:29.119073 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b681b-e126-4e38-9cd4-7367c8f5b780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:29.120157 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b681b-e126-4e38-9cd4-7367c8f5b780 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:29.120600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b681b-e126-4e38-9cd4-7367c8f5b780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:29.121015 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b681b-e126-4e38-9cd4-7367c8f5b780 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:29.122122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b681b-e126-4e38-9cd4-7367c8f5b780 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:29.124978 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b681b-e126-4e38-9cd4-7367c8f5b780 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:29.167425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1186/2384] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:40:29 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:29.171504 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e91be8-9f47-4eb6-aaa7-3d8577a320e2 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:29.182877 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e91be8-9f47-4eb6-aaa7-3d8577a320e2 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:29.187684 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182892ac-8064-4727-92f4-4f28754706b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:29.188388 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182892ac-8064-4727-92f4-4f28754706b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:29.189202 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182892ac-8064-4727-92f4-4f28754706b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:29.189644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182892ac-8064-4727-92f4-4f28754706b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:29.190446 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182892ac-8064-4727-92f4-4f28754706b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:29.192050 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182892ac-8064-4727-92f4-4f28754706b4 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:29.205056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1199/2385] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:40:29.210014 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b681b-e126-4e38-9cd4-7367c8f5b780 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:29.215526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:29.222391 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b681b-e126-4e38-9cd4-7367c8f5b780 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:29.229301 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be45f9-ce26-4fec-8638-bd032ff2a1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:29.230344 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be45f9-ce26-4fec-8638-bd032ff2a1a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:29.230521 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be45f9-ce26-4fec-8638-bd032ff2a1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:29.230813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be45f9-ce26-4fec-8638-bd032ff2a1a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:29.231618 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be45f9-ce26-4fec-8638-bd032ff2a1a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:29.233273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be45f9-ce26-4fec-8638-bd032ff2a1a6 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:29.245723 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1187/2386] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:40:29.250230 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182892ac-8064-4727-92f4-4f28754706b4 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:29.262322 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182892ac-8064-4727-92f4-4f28754706b4 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:29.267079 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45097ad5-aa7e-4e48-ba43-1e43485cb984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:29.268002 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45097ad5-aa7e-4e48-ba43-1e43485cb984 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:29.268293 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45097ad5-aa7e-4e48-ba43-1e43485cb984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:29.268578 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45097ad5-aa7e-4e48-ba43-1e43485cb984 admin admin] PATH_INFO: `/v3/projects/1989db9416914dfab7a6add951a0d442/users/d29f658460434ce0b7b3d744a44b1e1b/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:29.286379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45097ad5-aa7e-4e48-ba43-1e43485cb984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1989db9416914dfab7a6add951a0d442, user_id=d29f658460434ce0b7b3d744a44b1e1b, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:29.290113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45097ad5-aa7e-4e48-ba43-1e43485cb984 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:29.305897 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1200/2387] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:40:29 2022] PUT /identity/v3/projects/1989db9416914dfab7a6add951a0d442/users/d29f658460434ce0b7b3d744a44b1e1b/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:29.324802 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073482b-eab3-465e-be1c-6d2f5a8c2574 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:29.325100 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073482b-eab3-465e-be1c-6d2f5a8c2574 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:29.325409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073482b-eab3-465e-be1c-6d2f5a8c2574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:29.353335 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-9073482b-eab3-465e-be1c-6d2f5a8c2574 None None] MFA Rules not processed for user `d29f658460434ce0b7b3d744a44b1e1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:29.354565 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9073482b-eab3-465e-be1c-6d2f5a8c2574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:29.436269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1188/2388] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:29.440910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45097ad5-aa7e-4e48-ba43-1e43485cb984 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:29.499753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:29.527225 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45097ad5-aa7e-4e48-ba43-1e43485cb984 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:29.535997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6aff12-53fd-41d3-b0b9-fbe2e4a7da06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:29.537256 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6aff12-53fd-41d3-b0b9-fbe2e4a7da06 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:29.537710 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6aff12-53fd-41d3-b0b9-fbe2e4a7da06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:29.538191 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6aff12-53fd-41d3-b0b9-fbe2e4a7da06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:29.539346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6aff12-53fd-41d3-b0b9-fbe2e4a7da06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:29.541583 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6aff12-53fd-41d3-b0b9-fbe2e4a7da06 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:29.571467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1201/2389] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:40:29 2022] POST /identity/v3/projects => generated 375 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:29.575461 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073482b-eab3-465e-be1c-6d2f5a8c2574 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:29.589273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073482b-eab3-465e-be1c-6d2f5a8c2574 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:29.651719 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c54221-9fe8-416c-b919-99806a1433e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:29.652408 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c54221-9fe8-416c-b919-99806a1433e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:29.652726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c54221-9fe8-416c-b919-99806a1433e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:29.653009 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c54221-9fe8-416c-b919-99806a1433e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:29.653899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c54221-9fe8-416c-b919-99806a1433e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:29.655889 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c54221-9fe8-416c-b919-99806a1433e3 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:29.677401 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1189/2390] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:40:29 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:29.681817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6aff12-53fd-41d3-b0b9-fbe2e4a7da06 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:29.693843 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6aff12-53fd-41d3-b0b9-fbe2e4a7da06 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:29.698896 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec78e2-51f9-43f4-a696-19681c22d345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:29.699584 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec78e2-51f9-43f4-a696-19681c22d345 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:29.699877 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec78e2-51f9-43f4-a696-19681c22d345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:29.700161 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec78e2-51f9-43f4-a696-19681c22d345 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:29.700912 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec78e2-51f9-43f4-a696-19681c22d345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:29.702620 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec78e2-51f9-43f4-a696-19681c22d345 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:29.714601 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1202/2391] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:40:29.719539 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c54221-9fe8-416c-b919-99806a1433e3 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:29.730726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c54221-9fe8-416c-b919-99806a1433e3 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:29.735830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04723c-1e1a-4623-860d-00099f35b680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:29.736764 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04723c-1e1a-4623-860d-00099f35b680 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:29.737053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04723c-1e1a-4623-860d-00099f35b680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:29.737381 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04723c-1e1a-4623-860d-00099f35b680 admin admin] PATH_INFO: `/v3/projects/71ba81c5abdd4371a111a8ff54486e3b/users/ba75dba660774ebbb140812c977c1ce7/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:29.756326 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04723c-1e1a-4623-860d-00099f35b680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71ba81c5abdd4371a111a8ff54486e3b, user_id=ba75dba660774ebbb140812c977c1ce7, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:29.759111 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04723c-1e1a-4623-860d-00099f35b680 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:29.779375 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1190/2392] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:40:29 2022] PUT /identity/v3/projects/71ba81c5abdd4371a111a8ff54486e3b/users/ba75dba660774ebbb140812c977c1ce7/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:29.783791 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec78e2-51f9-43f4-a696-19681c22d345 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:29.794966 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec78e2-51f9-43f4-a696-19681c22d345 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:29.860017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c45feb-93d5-497e-92a0-496018eabc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:29.860706 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c45feb-93d5-497e-92a0-496018eabc78 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:29.860996 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c45feb-93d5-497e-92a0-496018eabc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:29.861311 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c45feb-93d5-497e-92a0-496018eabc78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:29.862059 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c45feb-93d5-497e-92a0-496018eabc78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:29.863663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c45feb-93d5-497e-92a0-496018eabc78 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:29.874530 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1203/2393] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:40:29.879839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04723c-1e1a-4623-860d-00099f35b680 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:29.892048 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04723c-1e1a-4623-860d-00099f35b680 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:29.898384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6faa1-bc28-4e15-96ca-0c5c18c1cd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:29.899303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6faa1-bc28-4e15-96ca-0c5c18c1cd34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:29.899591 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6faa1-bc28-4e15-96ca-0c5c18c1cd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:29.899965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6faa1-bc28-4e15-96ca-0c5c18c1cd34 admin admin] PATH_INFO: `/v3/projects/71ba81c5abdd4371a111a8ff54486e3b/users/ba75dba660774ebbb140812c977c1ce7/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:29.902009 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6faa1-bc28-4e15-96ca-0c5c18c1cd34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71ba81c5abdd4371a111a8ff54486e3b, user_id=ba75dba660774ebbb140812c977c1ce7, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:29.904652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6faa1-bc28-4e15-96ca-0c5c18c1cd34 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:29.917349 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1191/2394] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:40:29 2022] PUT /identity/v3/projects/71ba81c5abdd4371a111a8ff54486e3b/users/ba75dba660774ebbb140812c977c1ce7/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:40:29.927755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2f9d7-3f28-45f4-9b89-98468dd9bc48 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:29.928047 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2f9d7-3f28-45f4-9b89-98468dd9bc48 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:29.928333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2f9d7-3f28-45f4-9b89-98468dd9bc48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:29.957300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-4fc2f9d7-3f28-45f4-9b89-98468dd9bc48 None None] MFA Rules not processed for user `ba75dba660774ebbb140812c977c1ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:29.959274 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4fc2f9d7-3f28-45f4-9b89-98468dd9bc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:30.034100 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1204/2395] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:30.045428 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6faa1-bc28-4e15-96ca-0c5c18c1cd34 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:30.047172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f6c6faa1-bc28-4e15-96ca-0c5c18c1cd34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:30.092096 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:30.124562 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6faa1-bc28-4e15-96ca-0c5c18c1cd34 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:30.131065 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a1e42-bd91-486b-b5bd-3b02cdf8a3e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:30.132134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a1e42-bd91-486b-b5bd-3b02cdf8a3e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:30.132608 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a1e42-bd91-486b-b5bd-3b02cdf8a3e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:30.133069 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a1e42-bd91-486b-b5bd-3b02cdf8a3e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:30.207160 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a1e42-bd91-486b-b5bd-3b02cdf8a3e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:30.211291 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a1e42-bd91-486b-b5bd-3b02cdf8a3e3 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:30.222862 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1192/2396] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:40:32.871126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2f9d7-3f28-45f4-9b89-98468dd9bc48 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:32.923729 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:32.948547 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2f9d7-3f28-45f4-9b89-98468dd9bc48 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:32.965359 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ccb09-9f69-4d26-8b81-01ec43215465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:32.966375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ccb09-9f69-4d26-8b81-01ec43215465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:32.966666 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ccb09-9f69-4d26-8b81-01ec43215465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:32.966954 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ccb09-9f69-4d26-8b81-01ec43215465 admin admin] PATH_INFO: `/v3/users/d29f658460434ce0b7b3d744a44b1e1b` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:32.968367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ccb09-9f69-4d26-8b81-01ec43215465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d29f658460434ce0b7b3d744a44b1e1b)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:32.970356 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ccb09-9f69-4d26-8b81-01ec43215465 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:33.031223 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-4f5ccb09-9f69-4d26-8b81-01ec43215465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd29f658460434ce0b7b3d744a44b1e1b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:33.042524 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-4f5ccb09-9f69-4d26-8b81-01ec43215465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd29f658460434ce0b7b3d744a44b1e1b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:33.043280 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-4f5ccb09-9f69-4d26-8b81-01ec43215465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd29f658460434ce0b7b3d744a44b1e1b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:33.052600 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-4f5ccb09-9f69-4d26-8b81-01ec43215465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd29f658460434ce0b7b3d744a44b1e1b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:33.070690 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-4f5ccb09-9f69-4d26-8b81-01ec43215465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd29f658460434ce0b7b3d744a44b1e1b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:33.078743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1205/2397] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:40:32 2022] DELETE /identity/v3/users/d29f658460434ce0b7b3d744a44b1e1b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:33.084706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc0909-311a-43a2-bc20-e82004740088 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:33.085225 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc0909-311a-43a2-bc20-e82004740088 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:33.085225 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc0909-311a-43a2-bc20-e82004740088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:33.102927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-59cc0909-311a-43a2-bc20-e82004740088 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:33.104134 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-59cc0909-311a-43a2-bc20-e82004740088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:33.165689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1193/2398] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:33.175026 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ccb09-9f69-4d26-8b81-01ec43215465 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:33.176445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4f5ccb09-9f69-4d26-8b81-01ec43215465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:33.245958 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ccb09-9f69-4d26-8b81-01ec43215465 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:33.252122 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd1368-b893-4187-a2c4-b7e4ff1e6368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:33.253320 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd1368-b893-4187-a2c4-b7e4ff1e6368 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:33.253827 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd1368-b893-4187-a2c4-b7e4ff1e6368 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:33.254319 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd1368-b893-4187-a2c4-b7e4ff1e6368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:33.281830 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd1368-b893-4187-a2c4-b7e4ff1e6368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:33.283821 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd1368-b893-4187-a2c4-b7e4ff1e6368 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:33.295287 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1206/2399] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:40:33.681710 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc0909-311a-43a2-bc20-e82004740088 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:33.683942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-59cc0909-311a-43a2-bc20-e82004740088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:33.701789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc0909-311a-43a2-bc20-e82004740088 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:33.710368 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4943c-0983-4259-b6a0-2534b6426ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:33.711282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4943c-0983-4259-b6a0-2534b6426ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:33.711549 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4943c-0983-4259-b6a0-2534b6426ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:33.711826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4943c-0983-4259-b6a0-2534b6426ea6 admin admin] PATH_INFO: `/v3/projects/1989db9416914dfab7a6add951a0d442` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:33.713078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4943c-0983-4259-b6a0-2534b6426ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1989db9416914dfab7a6add951a0d442)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:33.715030 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4943c-0983-4259-b6a0-2534b6426ea6 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:33.805763 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-05f4943c-0983-4259-b6a0-2534b6426ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1989db9416914dfab7a6add951a0d442', 'request_id': 'req-05f4943c-0983-4259-b6a0-2534b6426ea6'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:33.816451 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-05f4943c-0983-4259-b6a0-2534b6426ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1989db9416914dfab7a6add951a0d442', 'request_id': 'req-05f4943c-0983-4259-b6a0-2534b6426ea6'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:33.824608 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-05f4943c-0983-4259-b6a0-2534b6426ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1989db9416914dfab7a6add951a0d442', 'request_id': 'req-05f4943c-0983-4259-b6a0-2534b6426ea6'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:33.838574 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-05f4943c-0983-4259-b6a0-2534b6426ea6 admin admin] The token cache is being invalidate because project 1989db9416914dfab7a6add951a0d442 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:40:33.839067 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-05f4943c-0983-4259-b6a0-2534b6426ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:33.840131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-05f4943c-0983-4259-b6a0-2534b6426ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:33.842933 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1194/2400] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:40:33 2022] DELETE /identity/v3/projects/1989db9416914dfab7a6add951a0d442 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:33.846943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd1368-b893-4187-a2c4-b7e4ff1e6368 service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:33.858321 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd1368-b893-4187-a2c4-b7e4ff1e6368 service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:33.927236 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384bd15-4736-40e2-aaf4-602fe6ce8457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:33.928112 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384bd15-4736-40e2-aaf4-602fe6ce8457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:33.928406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384bd15-4736-40e2-aaf4-602fe6ce8457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:33.928691 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384bd15-4736-40e2-aaf4-602fe6ce8457 admin admin] PATH_INFO: `/v3/users/ba75dba660774ebbb140812c977c1ce7` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:33.930383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384bd15-4736-40e2-aaf4-602fe6ce8457 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba75dba660774ebbb140812c977c1ce7)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:33.932367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384bd15-4736-40e2-aaf4-602fe6ce8457 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:33.935232 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:33.991709 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-8384bd15-4736-40e2-aaf4-602fe6ce8457 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba75dba660774ebbb140812c977c1ce7'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:33.999565 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-8384bd15-4736-40e2-aaf4-602fe6ce8457 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba75dba660774ebbb140812c977c1ce7'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:34.000359 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-8384bd15-4736-40e2-aaf4-602fe6ce8457 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba75dba660774ebbb140812c977c1ce7'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:34.009280 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-8384bd15-4736-40e2-aaf4-602fe6ce8457 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba75dba660774ebbb140812c977c1ce7'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:34.023230 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-8384bd15-4736-40e2-aaf4-602fe6ce8457 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba75dba660774ebbb140812c977c1ce7'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:34.029291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1207/2401] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:40:33 2022] DELETE /identity/v3/users/ba75dba660774ebbb140812c977c1ce7 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:34.326779 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4943c-0983-4259-b6a0-2534b6426ea6 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:34.328483 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-05f4943c-0983-4259-b6a0-2534b6426ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:34.343927 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4943c-0983-4259-b6a0-2534b6426ea6 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:34.352958 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:34.408632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f410155-723c-4d72-9ebf-46c0f191d823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:34.409544 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f410155-723c-4d72-9ebf-46c0f191d823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:34.409835 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f410155-723c-4d72-9ebf-46c0f191d823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:34.410117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f410155-723c-4d72-9ebf-46c0f191d823 admin admin] PATH_INFO: `/v3/projects/71ba81c5abdd4371a111a8ff54486e3b` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:34.411968 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f410155-723c-4d72-9ebf-46c0f191d823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71ba81c5abdd4371a111a8ff54486e3b)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:34.413630 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f410155-723c-4d72-9ebf-46c0f191d823 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:34.462818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4f410155-723c-4d72-9ebf-46c0f191d823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71ba81c5abdd4371a111a8ff54486e3b', 'request_id': 'req-4f410155-723c-4d72-9ebf-46c0f191d823'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:34.470768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4f410155-723c-4d72-9ebf-46c0f191d823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71ba81c5abdd4371a111a8ff54486e3b', 'request_id': 'req-4f410155-723c-4d72-9ebf-46c0f191d823'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:34.479884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4f410155-723c-4d72-9ebf-46c0f191d823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71ba81c5abdd4371a111a8ff54486e3b', 'request_id': 'req-4f410155-723c-4d72-9ebf-46c0f191d823'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:34.492501 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4f410155-723c-4d72-9ebf-46c0f191d823 admin admin] The token cache is being invalidate because project 71ba81c5abdd4371a111a8ff54486e3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:40:34.492936 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4f410155-723c-4d72-9ebf-46c0f191d823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:34.493640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4f410155-723c-4d72-9ebf-46c0f191d823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:34.495220 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1195/2402] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:40:34 2022] DELETE /identity/v3/projects/71ba81c5abdd4371a111a8ff54486e3b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:34.500125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec97c19-a45f-44c6-8726-b7258db435af None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:34.500453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec97c19-a45f-44c6-8726-b7258db435af None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:34.500764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec97c19-a45f-44c6-8726-b7258db435af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:34.521105 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-8ec97c19-a45f-44c6-8726-b7258db435af None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:34.522896 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-8ec97c19-a45f-44c6-8726-b7258db435af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:34.581850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1208/2403] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:34.589458 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c76534-1a18-4ed9-963a-a08463e2345e None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:34.589753 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c76534-1a18-4ed9-963a-a08463e2345e None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:34.590040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c76534-1a18-4ed9-963a-a08463e2345e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:34.609633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-14c76534-1a18-4ed9-963a-a08463e2345e None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:34.610847 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-14c76534-1a18-4ed9-963a-a08463e2345e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:34.620282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1196/2404] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:34.624867 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec97c19-a45f-44c6-8726-b7258db435af None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:34.634802 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec97c19-a45f-44c6-8726-b7258db435af None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:34.639357 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c126844-a698-41eb-bfd5-013f8f94cf6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:34.640014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c126844-a698-41eb-bfd5-013f8f94cf6d admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:34.640303 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c126844-a698-41eb-bfd5-013f8f94cf6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:34.640585 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c126844-a698-41eb-bfd5-013f8f94cf6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:34.641352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c126844-a698-41eb-bfd5-013f8f94cf6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:40:34.641669 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c126844-a698-41eb-bfd5-013f8f94cf6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:34.643288 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c126844-a698-41eb-bfd5-013f8f94cf6d admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:34.655718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1209/2405] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:40:34.660036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c76534-1a18-4ed9-963a-a08463e2345e None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:34.670107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c76534-1a18-4ed9-963a-a08463e2345e None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:34.674722 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ae445-8a13-4e89-9962-7e7d88c15a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:34.675380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ae445-8a13-4e89-9962-7e7d88c15a23 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:34.675672 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ae445-8a13-4e89-9962-7e7d88c15a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:34.676005 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ae445-8a13-4e89-9962-7e7d88c15a23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:34.677872 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ae445-8a13-4e89-9962-7e7d88c15a23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:34.679942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ae445-8a13-4e89-9962-7e7d88c15a23 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:34.716532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1197/2406] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:40:34 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:34.721680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c126844-a698-41eb-bfd5-013f8f94cf6d admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:34.734087 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c126844-a698-41eb-bfd5-013f8f94cf6d admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:34.804771 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97639c8f-a685-4a73-8e83-e0e306344b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:34.805618 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97639c8f-a685-4a73-8e83-e0e306344b4b admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:34.805916 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97639c8f-a685-4a73-8e83-e0e306344b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:34.806204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97639c8f-a685-4a73-8e83-e0e306344b4b admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:34.806922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97639c8f-a685-4a73-8e83-e0e306344b4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:34.808858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97639c8f-a685-4a73-8e83-e0e306344b4b admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:34.825463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:34.832412 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1210/2407] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:40:34 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:34.836842 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ae445-8a13-4e89-9962-7e7d88c15a23 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:34.850919 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ae445-8a13-4e89-9962-7e7d88c15a23 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:34.857164 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e05832-2554-4ec2-8d60-56b50eed60c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:34.857851 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e05832-2554-4ec2-8d60-56b50eed60c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:34.858141 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e05832-2554-4ec2-8d60-56b50eed60c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:34.858424 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e05832-2554-4ec2-8d60-56b50eed60c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:34.859164 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e05832-2554-4ec2-8d60-56b50eed60c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:34.861032 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e05832-2554-4ec2-8d60-56b50eed60c1 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:34.872958 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1198/2408] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:40:34.876926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97639c8f-a685-4a73-8e83-e0e306344b4b admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:34.888029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97639c8f-a685-4a73-8e83-e0e306344b4b admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:34.893091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d28b33-fc43-4ba5-bf73-bfb3f40b05c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:34.893806 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d28b33-fc43-4ba5-bf73-bfb3f40b05c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:34.894099 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d28b33-fc43-4ba5-bf73-bfb3f40b05c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:34.894384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d28b33-fc43-4ba5-bf73-bfb3f40b05c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:34.895120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d28b33-fc43-4ba5-bf73-bfb3f40b05c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:34.897446 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d28b33-fc43-4ba5-bf73-bfb3f40b05c3 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:34.908076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1211/2409] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:40:34.912834 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e05832-2554-4ec2-8d60-56b50eed60c1 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:34.923333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e05832-2554-4ec2-8d60-56b50eed60c1 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:34.928626 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50f85a-bdd3-478b-87a3-6be05b71b462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:34.929550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50f85a-bdd3-478b-87a3-6be05b71b462 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:34.929841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50f85a-bdd3-478b-87a3-6be05b71b462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:34.930122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50f85a-bdd3-478b-87a3-6be05b71b462 admin admin] PATH_INFO: `/v3/projects/1b82e66a68d948a48acc4822eaa493cb/users/242693c7e4bd454183d050c5fea5df70/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:34.948022 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c50f85a-bdd3-478b-87a3-6be05b71b462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b82e66a68d948a48acc4822eaa493cb, user_id=242693c7e4bd454183d050c5fea5df70, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:34.951007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c50f85a-bdd3-478b-87a3-6be05b71b462 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:34.985396 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1199/2410] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:40:34 2022] PUT /identity/v3/projects/1b82e66a68d948a48acc4822eaa493cb/users/242693c7e4bd454183d050c5fea5df70/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:34.988899 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3417bc4-8460-4d47-ac0a-ad1326de4ebc None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:34.989558 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3417bc4-8460-4d47-ac0a-ad1326de4ebc None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:34.989655 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3417bc4-8460-4d47-ac0a-ad1326de4ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:35.043891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-d3417bc4-8460-4d47-ac0a-ad1326de4ebc None None] MFA Rules not processed for user `242693c7e4bd454183d050c5fea5df70`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:35.045794 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-d3417bc4-8460-4d47-ac0a-ad1326de4ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:35.191354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1212/2411] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:35.206761 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50f85a-bdd3-478b-87a3-6be05b71b462 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:35.210921 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0c50f85a-bdd3-478b-87a3-6be05b71b462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:35.278011 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:35.306121 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50f85a-bdd3-478b-87a3-6be05b71b462 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:35.311343 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fbf05-f1a8-4c01-9888-7befd481a5af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:35.312023 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fbf05-f1a8-4c01-9888-7befd481a5af service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:35.312315 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fbf05-f1a8-4c01-9888-7befd481a5af service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:35.312598 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fbf05-f1a8-4c01-9888-7befd481a5af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:35.322789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276fbf05-f1a8-4c01-9888-7befd481a5af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:35.324920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276fbf05-f1a8-4c01-9888-7befd481a5af service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:35.336328 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90909a5-2b21-4482-829c-7238a9279ee8 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:35.336543 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1200/2412] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:40:35.337054 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90909a5-2b21-4482-829c-7238a9279ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:35.337562 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90909a5-2b21-4482-829c-7238a9279ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:35.366971 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-f90909a5-2b21-4482-829c-7238a9279ee8 None None] MFA Rules not processed for user `cfc2f31f654e486d8505d80aad17d17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:35.368478 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-f90909a5-2b21-4482-829c-7238a9279ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:35.447175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1213/2413] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 17:40:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:35.504887 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fbf05-f1a8-4c01-9888-7befd481a5af service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:35.507792 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-276fbf05-f1a8-4c01-9888-7befd481a5af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:35.606781 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fbf05-f1a8-4c01-9888-7befd481a5af service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:35.613577 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52779d65-a156-4762-9fc2-0cc4bbf18622 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a37d1201d54afdad9b228ef4f2d787', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:35.614946 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52779d65-a156-4762-9fc2-0cc4bbf18622 service swift] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:35.615259 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52779d65-a156-4762-9fc2-0cc4bbf18622 service swift] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:35.615762 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52779d65-a156-4762-9fc2-0cc4bbf18622 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:35.636247 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52779d65-a156-4762-9fc2-0cc4bbf18622 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:35.638829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52779d65-a156-4762-9fc2-0cc4bbf18622 service swift] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:35.652662 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1201/2414] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 17:40:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:40:38.691211 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90909a5-2b21-4482-829c-7238a9279ee8 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:38.812651 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90909a5-2b21-4482-829c-7238a9279ee8 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:38.818949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc0673-9d7f-435a-8ce0-8d405b13c047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:38.819932 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc0673-9d7f-435a-8ce0-8d405b13c047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:38.820236 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc0673-9d7f-435a-8ce0-8d405b13c047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:38.820532 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc0673-9d7f-435a-8ce0-8d405b13c047 admin admin] PATH_INFO: `/v3/users/242693c7e4bd454183d050c5fea5df70` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:38.821913 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc0673-9d7f-435a-8ce0-8d405b13c047 admin admin] RBAC: Authorizing `identity:delete_user(user_id=242693c7e4bd454183d050c5fea5df70)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:38.824139 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc0673-9d7f-435a-8ce0-8d405b13c047 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:38.852626 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:38.887929 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d6dc0673-9d7f-435a-8ce0-8d405b13c047 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '242693c7e4bd454183d050c5fea5df70'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:38.903174 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d6dc0673-9d7f-435a-8ce0-8d405b13c047 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '242693c7e4bd454183d050c5fea5df70'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:38.906928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d6dc0673-9d7f-435a-8ce0-8d405b13c047 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '242693c7e4bd454183d050c5fea5df70'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:38.916513 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d6dc0673-9d7f-435a-8ce0-8d405b13c047 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '242693c7e4bd454183d050c5fea5df70'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:38.935205 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d6dc0673-9d7f-435a-8ce0-8d405b13c047 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '242693c7e4bd454183d050c5fea5df70'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:38.943655 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1214/2415] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:40:38 2022] DELETE /identity/v3/users/242693c7e4bd454183d050c5fea5df70 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:40:38.948740 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2c2af-f930-4cc1-a574-b6cccc0cdec1 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:38.949358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2c2af-f930-4cc1-a574-b6cccc0cdec1 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:38.949358 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2c2af-f930-4cc1-a574-b6cccc0cdec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:38.979800 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-46d2c2af-f930-4cc1-a574-b6cccc0cdec1 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:38.981385 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-46d2c2af-f930-4cc1-a574-b6cccc0cdec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:39.098896 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1202/2416] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:40:39.110559 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc0673-9d7f-435a-8ce0-8d405b13c047 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:39.116728 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-d6dc0673-9d7f-435a-8ce0-8d405b13c047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:39.213769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc0673-9d7f-435a-8ce0-8d405b13c047 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:39.220053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2651b-c7dd-4ab5-9b98-4e3ae0b816fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:39.221113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2651b-c7dd-4ab5-9b98-4e3ae0b816fa service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:39.221735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2651b-c7dd-4ab5-9b98-4e3ae0b816fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:39.222299 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2651b-c7dd-4ab5-9b98-4e3ae0b816fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:39.241970 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d2651b-c7dd-4ab5-9b98-4e3ae0b816fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:39.243437 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d2651b-c7dd-4ab5-9b98-4e3ae0b816fa service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:39.255747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1215/2417] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:40:39.671179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2c2af-f930-4cc1-a574-b6cccc0cdec1 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:39.673458 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-46d2c2af-f930-4cc1-a574-b6cccc0cdec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:39.709990 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2c2af-f930-4cc1-a574-b6cccc0cdec1 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:39.721664 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda8ccf-b387-4b82-b283-2a50c467d85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:39.722578 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda8ccf-b387-4b82-b283-2a50c467d85a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:39.722875 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda8ccf-b387-4b82-b283-2a50c467d85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:39.723777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda8ccf-b387-4b82-b283-2a50c467d85a admin admin] PATH_INFO: `/v3/projects/1b82e66a68d948a48acc4822eaa493cb` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:39.730903 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda8ccf-b387-4b82-b283-2a50c467d85a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b82e66a68d948a48acc4822eaa493cb)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:39.735692 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda8ccf-b387-4b82-b283-2a50c467d85a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:39.854027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-deda8ccf-b387-4b82-b283-2a50c467d85a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b82e66a68d948a48acc4822eaa493cb', 'request_id': 'req-deda8ccf-b387-4b82-b283-2a50c467d85a'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:39.872756 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-deda8ccf-b387-4b82-b283-2a50c467d85a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b82e66a68d948a48acc4822eaa493cb', 'request_id': 'req-deda8ccf-b387-4b82-b283-2a50c467d85a'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:39.881899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-deda8ccf-b387-4b82-b283-2a50c467d85a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b82e66a68d948a48acc4822eaa493cb', 'request_id': 'req-deda8ccf-b387-4b82-b283-2a50c467d85a'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:39.888057 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:39.904623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-deda8ccf-b387-4b82-b283-2a50c467d85a admin admin] The token cache is being invalidate because project 1b82e66a68d948a48acc4822eaa493cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:40:39.906007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-deda8ccf-b387-4b82-b283-2a50c467d85a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:39.913012 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-deda8ccf-b387-4b82-b283-2a50c467d85a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:39.914813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1203/2418] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:40:39 2022] DELETE /identity/v3/projects/1b82e66a68d948a48acc4822eaa493cb => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:39.920813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17992b-f97b-4625-9358-25bc5a4749e9 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:39.921397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17992b-f97b-4625-9358-25bc5a4749e9 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:39.921397 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17992b-f97b-4625-9358-25bc5a4749e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:39.949807 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-2e17992b-f97b-4625-9358-25bc5a4749e9 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:39.951121 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-2e17992b-f97b-4625-9358-25bc5a4749e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:40.052676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1216/2419] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:40.062335 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d4068-5fde-480b-920e-6403a095125d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:40.062966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d4068-5fde-480b-920e-6403a095125d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:40.062966 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d4068-5fde-480b-920e-6403a095125d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:40.099734 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-2a5d4068-5fde-480b-920e-6403a095125d None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:40.101676 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2a5d4068-5fde-480b-920e-6403a095125d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:40.119851 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1204/2420] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:40.126831 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17992b-f97b-4625-9358-25bc5a4749e9 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:40.151848 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17992b-f97b-4625-9358-25bc5a4749e9 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:40.160083 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556cd5cc-a672-4635-8f19-e67aebd6b67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:40.162677 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556cd5cc-a672-4635-8f19-e67aebd6b67c admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:40.163166 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556cd5cc-a672-4635-8f19-e67aebd6b67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:40.163647 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556cd5cc-a672-4635-8f19-e67aebd6b67c admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:40.165032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556cd5cc-a672-4635-8f19-e67aebd6b67c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:40:40.165593 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556cd5cc-a672-4635-8f19-e67aebd6b67c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:40.168348 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556cd5cc-a672-4635-8f19-e67aebd6b67c admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:40.195057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1217/2421] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:40:40.202524 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d4068-5fde-480b-920e-6403a095125d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:40.221016 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d4068-5fde-480b-920e-6403a095125d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:40.237748 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765424e-1c62-4e74-b022-082a00c63816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:40.238864 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765424e-1c62-4e74-b022-082a00c63816 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:40.240405 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765424e-1c62-4e74-b022-082a00c63816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:40.240889 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765424e-1c62-4e74-b022-082a00c63816 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:40.242177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1765424e-1c62-4e74-b022-082a00c63816 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:40.245699 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1765424e-1c62-4e74-b022-082a00c63816 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:40.352131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1205/2422] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:40:40 2022] POST /identity/v3/projects => generated 383 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:40:40.358592 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556cd5cc-a672-4635-8f19-e67aebd6b67c admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:40.389341 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556cd5cc-a672-4635-8f19-e67aebd6b67c admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:40.489988 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:40.579219 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee362a0-dce6-4b99-823c-dc3114fe8b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:40.580872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee362a0-dce6-4b99-823c-dc3114fe8b5d admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:40.581922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee362a0-dce6-4b99-823c-dc3114fe8b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:40.582902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee362a0-dce6-4b99-823c-dc3114fe8b5d admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:40.586091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee362a0-dce6-4b99-823c-dc3114fe8b5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:40.589463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee362a0-dce6-4b99-823c-dc3114fe8b5d admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:40.653684 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1218/2423] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:40:40 2022] POST /identity/v3/users => generated 292 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:40.656921 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765424e-1c62-4e74-b022-082a00c63816 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:40.680327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765424e-1c62-4e74-b022-082a00c63816 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:40.699949 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd06599-192d-4909-b0eb-3242f0f37e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:40.701386 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd06599-192d-4909-b0eb-3242f0f37e83 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:40.701386 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd06599-192d-4909-b0eb-3242f0f37e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:40.701386 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd06599-192d-4909-b0eb-3242f0f37e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:40.702066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd06599-192d-4909-b0eb-3242f0f37e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:40.704399 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd06599-192d-4909-b0eb-3242f0f37e83 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:40.748640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1206/2424] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:40:40.763453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee362a0-dce6-4b99-823c-dc3114fe8b5d admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:40.800437 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee362a0-dce6-4b99-823c-dc3114fe8b5d admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:40.819052 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86f344-fbaf-4f96-ab76-5c0f890ca095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:40.820402 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86f344-fbaf-4f96-ab76-5c0f890ca095 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:40.820851 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86f344-fbaf-4f96-ab76-5c0f890ca095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:40.822794 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86f344-fbaf-4f96-ab76-5c0f890ca095 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:40.824086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f86f344-fbaf-4f96-ab76-5c0f890ca095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:40.828162 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f86f344-fbaf-4f96-ab76-5c0f890ca095 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:40.852927 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1219/2425] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:40:40.859969 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd06599-192d-4909-b0eb-3242f0f37e83 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:40.881222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd06599-192d-4909-b0eb-3242f0f37e83 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:40.899492 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ea5de-e075-40a4-a9b3-d02a91e1a3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:40.900964 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ea5de-e075-40a4-a9b3-d02a91e1a3cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:40.901541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ea5de-e075-40a4-a9b3-d02a91e1a3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:40.901996 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ea5de-e075-40a4-a9b3-d02a91e1a3cb admin admin] PATH_INFO: `/v3/projects/e581456dc52948ac9fecaeba0c2fcc28/users/70783c17bbed47cc8a479d8bd15f6378/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:40.926819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ea5de-e075-40a4-a9b3-d02a91e1a3cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e581456dc52948ac9fecaeba0c2fcc28, user_id=70783c17bbed47cc8a479d8bd15f6378, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:40.931232 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ea5de-e075-40a4-a9b3-d02a91e1a3cb admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:40.978273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1207/2426] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:40:40 2022] PUT /identity/v3/projects/e581456dc52948ac9fecaeba0c2fcc28/users/70783c17bbed47cc8a479d8bd15f6378/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:40.991560 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d86033-99bc-48ec-8177-8101b2443753 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:40.992181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d86033-99bc-48ec-8177-8101b2443753 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:40.992181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d86033-99bc-48ec-8177-8101b2443753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:41.029116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-43d86033-99bc-48ec-8177-8101b2443753 None None] MFA Rules not processed for user `70783c17bbed47cc8a479d8bd15f6378`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:41.030629 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-43d86033-99bc-48ec-8177-8101b2443753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:41.129009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1220/2427] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:41.147797 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ea5de-e075-40a4-a9b3-d02a91e1a3cb admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:41.152483 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e61ea5de-e075-40a4-a9b3-d02a91e1a3cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:41.266193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ea5de-e075-40a4-a9b3-d02a91e1a3cb admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:41.275935 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592aa58-e322-42d2-92ec-a8d318ed5fa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:41.277691 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592aa58-e322-42d2-92ec-a8d318ed5fa9 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:41.278173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592aa58-e322-42d2-92ec-a8d318ed5fa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:41.278643 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592aa58-e322-42d2-92ec-a8d318ed5fa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:41.314919 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6592aa58-e322-42d2-92ec-a8d318ed5fa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:41.320500 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6592aa58-e322-42d2-92ec-a8d318ed5fa9 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:41.345766 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1208/2428] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:40:44.326662 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d86033-99bc-48ec-8177-8101b2443753 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:44.420253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d86033-99bc-48ec-8177-8101b2443753 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:44.426083 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8f03e-ac0f-41b1-83c3-917618bc12a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:44.428309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8f03e-ac0f-41b1-83c3-917618bc12a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:44.428309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8f03e-ac0f-41b1-83c3-917618bc12a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:44.428590 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8f03e-ac0f-41b1-83c3-917618bc12a3 admin admin] PATH_INFO: `/v3/users/70783c17bbed47cc8a479d8bd15f6378` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:44.430147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8f03e-ac0f-41b1-83c3-917618bc12a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70783c17bbed47cc8a479d8bd15f6378)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:44.432467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8f03e-ac0f-41b1-83c3-917618bc12a3 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:44.499929 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:44.508784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b9a8f03e-ac0f-41b1-83c3-917618bc12a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70783c17bbed47cc8a479d8bd15f6378'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:44.533528 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b9a8f03e-ac0f-41b1-83c3-917618bc12a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70783c17bbed47cc8a479d8bd15f6378'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:44.534325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b9a8f03e-ac0f-41b1-83c3-917618bc12a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70783c17bbed47cc8a479d8bd15f6378'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:44.545834 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b9a8f03e-ac0f-41b1-83c3-917618bc12a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70783c17bbed47cc8a479d8bd15f6378'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:44.565551 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-b9a8f03e-ac0f-41b1-83c3-917618bc12a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70783c17bbed47cc8a479d8bd15f6378'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:44.577536 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1221/2429] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:40:44 2022] DELETE /identity/v3/users/70783c17bbed47cc8a479d8bd15f6378 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:44.578592 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce5f86-d213-492a-8b31-bd1d14b3757d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:44.579054 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce5f86-d213-492a-8b31-bd1d14b3757d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:44.579137 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce5f86-d213-492a-8b31-bd1d14b3757d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:44.609630 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-9fce5f86-d213-492a-8b31-bd1d14b3757d None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:44.611286 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9fce5f86-d213-492a-8b31-bd1d14b3757d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:44.746793 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1209/2430] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:44.766509 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8f03e-ac0f-41b1-83c3-917618bc12a3 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:44.770807 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b9a8f03e-ac0f-41b1-83c3-917618bc12a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:44.870722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8f03e-ac0f-41b1-83c3-917618bc12a3 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:44.875315 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0636ee-90d3-4cdd-bccf-4ddb8d2406d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:44.875995 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0636ee-90d3-4cdd-bccf-4ddb8d2406d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:44.876284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0636ee-90d3-4cdd-bccf-4ddb8d2406d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:44.876637 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0636ee-90d3-4cdd-bccf-4ddb8d2406d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:44.899541 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0636ee-90d3-4cdd-bccf-4ddb8d2406d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:44.901383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0636ee-90d3-4cdd-bccf-4ddb8d2406d1 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:44.927374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1222/2431] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:40:45.389699 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce5f86-d213-492a-8b31-bd1d14b3757d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:45.393100 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9fce5f86-d213-492a-8b31-bd1d14b3757d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:45.407797 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce5f86-d213-492a-8b31-bd1d14b3757d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:45.414438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb3a4a-92eb-47f7-a30b-d545dd84c6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:45.416329 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb3a4a-92eb-47f7-a30b-d545dd84c6cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:45.416707 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb3a4a-92eb-47f7-a30b-d545dd84c6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:45.417067 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb3a4a-92eb-47f7-a30b-d545dd84c6cd admin admin] PATH_INFO: `/v3/projects/e581456dc52948ac9fecaeba0c2fcc28` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:45.418482 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb3a4a-92eb-47f7-a30b-d545dd84c6cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e581456dc52948ac9fecaeba0c2fcc28)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:45.435666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb3a4a-92eb-47f7-a30b-d545dd84c6cd admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:45.505202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:45.533605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-26eb3a4a-92eb-47f7-a30b-d545dd84c6cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e581456dc52948ac9fecaeba0c2fcc28', 'request_id': 'req-26eb3a4a-92eb-47f7-a30b-d545dd84c6cd'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:45.545332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-26eb3a4a-92eb-47f7-a30b-d545dd84c6cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e581456dc52948ac9fecaeba0c2fcc28', 'request_id': 'req-26eb3a4a-92eb-47f7-a30b-d545dd84c6cd'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:45.555600 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-26eb3a4a-92eb-47f7-a30b-d545dd84c6cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e581456dc52948ac9fecaeba0c2fcc28', 'request_id': 'req-26eb3a4a-92eb-47f7-a30b-d545dd84c6cd'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:45.575167 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-26eb3a4a-92eb-47f7-a30b-d545dd84c6cd admin admin] The token cache is being invalidate because project e581456dc52948ac9fecaeba0c2fcc28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:40:45.575612 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-26eb3a4a-92eb-47f7-a30b-d545dd84c6cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:45.577481 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-26eb3a4a-92eb-47f7-a30b-d545dd84c6cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:45.581535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1210/2432] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:40:45 2022] DELETE /identity/v3/projects/e581456dc52948ac9fecaeba0c2fcc28 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:45.587470 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc10813-a62b-42df-be9e-c272b1b19385 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:45.588318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc10813-a62b-42df-be9e-c272b1b19385 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:45.588768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc10813-a62b-42df-be9e-c272b1b19385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:45.622670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-8fc10813-a62b-42df-be9e-c272b1b19385 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:45.624019 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-8fc10813-a62b-42df-be9e-c272b1b19385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:45.736060 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1223/2433] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:45.744508 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ff970-4d51-4d23-9215-116c46572690 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:45.745057 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ff970-4d51-4d23-9215-116c46572690 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:45.745419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ff970-4d51-4d23-9215-116c46572690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:45.772491 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-de9ff970-4d51-4d23-9215-116c46572690 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:45.774084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-de9ff970-4d51-4d23-9215-116c46572690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:45.784266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1211/2434] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:45.789946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc10813-a62b-42df-be9e-c272b1b19385 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:45.802886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc10813-a62b-42df-be9e-c272b1b19385 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:45.808197 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff7bed-541d-43f0-ac77-598d9a974096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:45.809044 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff7bed-541d-43f0-ac77-598d9a974096 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:45.809375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff7bed-541d-43f0-ac77-598d9a974096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:45.809662 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff7bed-541d-43f0-ac77-598d9a974096 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:45.810415 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ff7bed-541d-43f0-ac77-598d9a974096 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:40:45.810738 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ff7bed-541d-43f0-ac77-598d9a974096 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:45.812448 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ff7bed-541d-43f0-ac77-598d9a974096 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:45.828943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1224/2435] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:40:45.833673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ff970-4d51-4d23-9215-116c46572690 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:45.849392 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ff970-4d51-4d23-9215-116c46572690 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:45.857713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90176ced-d45f-4a65-a6ea-96150adb684c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:45.858470 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90176ced-d45f-4a65-a6ea-96150adb684c admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:45.859231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90176ced-d45f-4a65-a6ea-96150adb684c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:45.859445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90176ced-d45f-4a65-a6ea-96150adb684c admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:45.860781 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90176ced-d45f-4a65-a6ea-96150adb684c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:45.872963 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90176ced-d45f-4a65-a6ea-96150adb684c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:45.930980 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1212/2436] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:40:45 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:45.935160 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff7bed-541d-43f0-ac77-598d9a974096 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:45.955111 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff7bed-541d-43f0-ac77-598d9a974096 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:46.053926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:46.093399 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c99af5-7db9-4dcc-b3fd-9deecd51ca23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:46.094113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c99af5-7db9-4dcc-b3fd-9deecd51ca23 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:46.094541 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c99af5-7db9-4dcc-b3fd-9deecd51ca23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:46.094905 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c99af5-7db9-4dcc-b3fd-9deecd51ca23 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:46.095680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c99af5-7db9-4dcc-b3fd-9deecd51ca23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:46.097942 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c99af5-7db9-4dcc-b3fd-9deecd51ca23 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:46.135081 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1225/2437] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:40:45 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:46.139020 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90176ced-d45f-4a65-a6ea-96150adb684c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:46.167774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90176ced-d45f-4a65-a6ea-96150adb684c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:46.178283 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c76466-b9de-4217-93c7-f802b75e21b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:46.179798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c76466-b9de-4217-93c7-f802b75e21b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:46.180095 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c76466-b9de-4217-93c7-f802b75e21b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:46.180388 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c76466-b9de-4217-93c7-f802b75e21b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:46.181223 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c76466-b9de-4217-93c7-f802b75e21b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:46.184091 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c76466-b9de-4217-93c7-f802b75e21b8 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:46.206466 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1213/2438] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:40:46.211807 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c99af5-7db9-4dcc-b3fd-9deecd51ca23 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:46.223840 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c99af5-7db9-4dcc-b3fd-9deecd51ca23 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:46.236510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838dc21-77af-45e6-9987-dc45cfe46f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:46.237225 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838dc21-77af-45e6-9987-dc45cfe46f61 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:46.238226 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838dc21-77af-45e6-9987-dc45cfe46f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:46.238226 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838dc21-77af-45e6-9987-dc45cfe46f61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:46.239018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838dc21-77af-45e6-9987-dc45cfe46f61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:46.240779 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838dc21-77af-45e6-9987-dc45cfe46f61 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:46.261646 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1226/2439] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:40:46.263770 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c76466-b9de-4217-93c7-f802b75e21b8 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:46.280694 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c76466-b9de-4217-93c7-f802b75e21b8 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:46.290486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec983098-93b5-498d-a381-5e060a724330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:46.291401 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec983098-93b5-498d-a381-5e060a724330 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:46.291684 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec983098-93b5-498d-a381-5e060a724330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:46.292011 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec983098-93b5-498d-a381-5e060a724330 admin admin] PATH_INFO: `/v3/projects/ad7c8fc06d764104b1c7107b48d8dddc/users/af7be60f65da43f982c1ec2e91397051/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:46.317416 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec983098-93b5-498d-a381-5e060a724330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad7c8fc06d764104b1c7107b48d8dddc, user_id=af7be60f65da43f982c1ec2e91397051, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:46.320146 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec983098-93b5-498d-a381-5e060a724330 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:46.346872 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1214/2440] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:40:46 2022] PUT /identity/v3/projects/ad7c8fc06d764104b1c7107b48d8dddc/users/af7be60f65da43f982c1ec2e91397051/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:46.352234 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4aaa46-9c7d-423f-9e28-5cbe85d95fbd None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:46.352786 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4aaa46-9c7d-423f-9e28-5cbe85d95fbd None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:46.353118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4aaa46-9c7d-423f-9e28-5cbe85d95fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:46.389235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-cb4aaa46-9c7d-423f-9e28-5cbe85d95fbd None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:46.389235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-cb4aaa46-9c7d-423f-9e28-5cbe85d95fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:46.515588 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1227/2441] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:46.558159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec983098-93b5-498d-a381-5e060a724330 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:46.562398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ec983098-93b5-498d-a381-5e060a724330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:46.643576 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec983098-93b5-498d-a381-5e060a724330 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:46.652453 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe90423-5d53-47c4-81e7-f5805a0c0431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:46.653366 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe90423-5d53-47c4-81e7-f5805a0c0431 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:46.653661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe90423-5d53-47c4-81e7-f5805a0c0431 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:46.653953 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe90423-5d53-47c4-81e7-f5805a0c0431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:46.668979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe90423-5d53-47c4-81e7-f5805a0c0431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:46.669416 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe90423-5d53-47c4-81e7-f5805a0c0431 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:46.681651 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1215/2442] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:40:47.127076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4aaa46-9c7d-423f-9e28-5cbe85d95fbd None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:47.128711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-cb4aaa46-9c7d-423f-9e28-5cbe85d95fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:47.318137 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4aaa46-9c7d-423f-9e28-5cbe85d95fbd None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:47.331030 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f98f21-9182-41f6-ad4c-bad891f2f969 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:47.333367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f98f21-9182-41f6-ad4c-bad891f2f969 service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:47.334112 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f98f21-9182-41f6-ad4c-bad891f2f969 service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:47.334476 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f98f21-9182-41f6-ad4c-bad891f2f969 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:47.340405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-10f98f21-9182-41f6-ad4c-bad891f2f969 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:47.482261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f98f21-9182-41f6-ad4c-bad891f2f969 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:47.486482 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f98f21-9182-41f6-ad4c-bad891f2f969 service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:47.531851 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1228/2443] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:40:48.787924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe90423-5d53-47c4-81e7-f5805a0c0431 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:48.789417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-dfe90423-5d53-47c4-81e7-f5805a0c0431 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:48.804633 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe90423-5d53-47c4-81e7-f5805a0c0431 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:48.809374 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c8191-f2a5-4710-be43-800ed25e9299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:48.810409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c8191-f2a5-4710-be43-800ed25e9299 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:48.810951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c8191-f2a5-4710-be43-800ed25e9299 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:48.811416 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c8191-f2a5-4710-be43-800ed25e9299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:48.814734 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5d4c8191-f2a5-4710-be43-800ed25e9299 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:48.832728 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c8191-f2a5-4710-be43-800ed25e9299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:48.835414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c8191-f2a5-4710-be43-800ed25e9299 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:48.848471 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1216/2444] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:40:52.415541 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009baf79-4eb5-4b6e-a005-3b1ca00b55af None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:52.416061 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009baf79-4eb5-4b6e-a005-3b1ca00b55af None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:52.416061 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009baf79-4eb5-4b6e-a005-3b1ca00b55af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:52.423881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:52.461723 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-009baf79-4eb5-4b6e-a005-3b1ca00b55af None None] MFA Rules not processed for user `af7be60f65da43f982c1ec2e91397051`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:52.463339 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-009baf79-4eb5-4b6e-a005-3b1ca00b55af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:52.585280 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1229/2445] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:40:52.587870 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c8191-f2a5-4710-be43-800ed25e9299 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:52.599893 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c8191-f2a5-4710-be43-800ed25e9299 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:52.605301 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494cc57-f3e6-4306-bb01-f5c212432c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:52.606111 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494cc57-f3e6-4306-bb01-f5c212432c84 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:52.606472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494cc57-f3e6-4306-bb01-f5c212432c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:52.606822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494cc57-f3e6-4306-bb01-f5c212432c84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:52.607780 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494cc57-f3e6-4306-bb01-f5c212432c84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:52.610426 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494cc57-f3e6-4306-bb01-f5c212432c84 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:52.655224 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1217/2446] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:40:52 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:52.656511 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009baf79-4eb5-4b6e-a005-3b1ca00b55af None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:52.682387 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009baf79-4eb5-4b6e-a005-3b1ca00b55af None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:52.805339 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624fda0-f7dd-4f77-8127-f53ecb11451e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:52.807858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624fda0-f7dd-4f77-8127-f53ecb11451e admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:52.807858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624fda0-f7dd-4f77-8127-f53ecb11451e admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:52.808093 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624fda0-f7dd-4f77-8127-f53ecb11451e admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:52.809962 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624fda0-f7dd-4f77-8127-f53ecb11451e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:52.813635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624fda0-f7dd-4f77-8127-f53ecb11451e admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:52.845234 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1230/2447] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:40:52 2022] POST /identity/v3/users => generated 298 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:52.850261 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494cc57-f3e6-4306-bb01-f5c212432c84 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:52.871776 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494cc57-f3e6-4306-bb01-f5c212432c84 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:52.882049 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cc6dc-31be-4b0e-af92-07cb6478b9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:52.883192 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cc6dc-31be-4b0e-af92-07cb6478b9ca admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:52.883794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cc6dc-31be-4b0e-af92-07cb6478b9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:52.884315 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cc6dc-31be-4b0e-af92-07cb6478b9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:52.885955 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cc6dc-31be-4b0e-af92-07cb6478b9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:52.887907 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cc6dc-31be-4b0e-af92-07cb6478b9ca admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:52.926128 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1218/2448] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:40:52.927543 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624fda0-f7dd-4f77-8127-f53ecb11451e admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:52.943563 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624fda0-f7dd-4f77-8127-f53ecb11451e admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:52.963673 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc45c7-3687-42a1-8025-cda52f066394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:52.965067 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc45c7-3687-42a1-8025-cda52f066394 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:52.965447 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc45c7-3687-42a1-8025-cda52f066394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:52.965738 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc45c7-3687-42a1-8025-cda52f066394 admin admin] PATH_INFO: `/v3/projects/0c9770e1404c439abaa86ffb75099000/users/7a3db4c2153340eeaf907c2fe0919e41/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:52.996142 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc45c7-3687-42a1-8025-cda52f066394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c9770e1404c439abaa86ffb75099000, user_id=7a3db4c2153340eeaf907c2fe0919e41, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:52.998920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc45c7-3687-42a1-8025-cda52f066394 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:53.017455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1231/2449] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:40:52 2022] PUT /identity/v3/projects/0c9770e1404c439abaa86ffb75099000/users/7a3db4c2153340eeaf907c2fe0919e41/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:40:53.022892 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cc6dc-31be-4b0e-af92-07cb6478b9ca admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:53.033593 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cc6dc-31be-4b0e-af92-07cb6478b9ca admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:53.040199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:53.116292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9202e-9456-4fa8-a8a3-1198fca6d155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:53.117530 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9202e-9456-4fa8-a8a3-1198fca6d155 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:53.118050 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9202e-9456-4fa8-a8a3-1198fca6d155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:53.118515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9202e-9456-4fa8-a8a3-1198fca6d155 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:53.119517 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae9202e-9456-4fa8-a8a3-1198fca6d155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:53.121431 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae9202e-9456-4fa8-a8a3-1198fca6d155 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:53.139947 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1219/2450] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:40:53.143901 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc45c7-3687-42a1-8025-cda52f066394 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:53.157612 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc45c7-3687-42a1-8025-cda52f066394 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:53.164224 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05da235-bd1e-4232-b4fd-56bef1a22a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:53.165309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05da235-bd1e-4232-b4fd-56bef1a22a48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:53.165694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05da235-bd1e-4232-b4fd-56bef1a22a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:53.166044 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05da235-bd1e-4232-b4fd-56bef1a22a48 admin admin] PATH_INFO: `/v3/projects/0c9770e1404c439abaa86ffb75099000/users/7a3db4c2153340eeaf907c2fe0919e41/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:53.168343 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05da235-bd1e-4232-b4fd-56bef1a22a48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c9770e1404c439abaa86ffb75099000, user_id=7a3db4c2153340eeaf907c2fe0919e41, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:53.171174 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05da235-bd1e-4232-b4fd-56bef1a22a48 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:53.188124 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1232/2451] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:40:53 2022] PUT /identity/v3/projects/0c9770e1404c439abaa86ffb75099000/users/7a3db4c2153340eeaf907c2fe0919e41/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:55.304551 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9202e-9456-4fa8-a8a3-1198fca6d155 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:55.306977 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-cae9202e-9456-4fa8-a8a3-1198fca6d155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:55.401088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05da235-bd1e-4232-b4fd-56bef1a22a48 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:55.404253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b05da235-bd1e-4232-b4fd-56bef1a22a48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:55.524435 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:55.529420 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9202e-9456-4fa8-a8a3-1198fca6d155 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:55.560418 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05da235-bd1e-4232-b4fd-56bef1a22a48 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:55.572900 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d07e06-a382-4b53-81af-40486da60d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:55.578917 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d07e06-a382-4b53-81af-40486da60d24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:55.579209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d07e06-a382-4b53-81af-40486da60d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:55.579475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d07e06-a382-4b53-81af-40486da60d24 admin admin] PATH_INFO: `/v3/users/3faab41439184fecbb3e1021eb81d555` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:55.609496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d07e06-a382-4b53-81af-40486da60d24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3faab41439184fecbb3e1021eb81d555)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:55.616236 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d07e06-a382-4b53-81af-40486da60d24 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:55.739096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c2d07e06-a382-4b53-81af-40486da60d24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3faab41439184fecbb3e1021eb81d555'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:55.778835 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d6a7d-417c-4b6b-8c5d-ff072fbe42c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75f0586a8c1941448d750918183f6dc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:55.784013 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c2d07e06-a382-4b53-81af-40486da60d24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3faab41439184fecbb3e1021eb81d555'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:55.785710 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d6a7d-417c-4b6b-8c5d-ff072fbe42c9 service placement] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:55.786173 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c2d07e06-a382-4b53-81af-40486da60d24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3faab41439184fecbb3e1021eb81d555'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:55.788265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d6a7d-417c-4b6b-8c5d-ff072fbe42c9 service placement] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:55.789174 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d6a7d-417c-4b6b-8c5d-ff072fbe42c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:55.796615 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c11d6a7d-417c-4b6b-8c5d-ff072fbe42c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:55.804542 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c2d07e06-a382-4b53-81af-40486da60d24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3faab41439184fecbb3e1021eb81d555'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:55.844176 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d6a7d-417c-4b6b-8c5d-ff072fbe42c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:55.845908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d6a7d-417c-4b6b-8c5d-ff072fbe42c9 service placement] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:55.869064 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c2d07e06-a382-4b53-81af-40486da60d24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3faab41439184fecbb3e1021eb81d555'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:55.873402 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1220/2452] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:40:55.895595 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1233/2453] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:40:55 2022] DELETE /identity/v3/users/3faab41439184fecbb3e1021eb81d555 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:56.310551 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d6a7d-417c-4b6b-8c5d-ff072fbe42c9 service placement] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:56.328923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d6a7d-417c-4b6b-8c5d-ff072fbe42c9 service placement] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:56.374576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bbc11-4570-47a3-a06a-e776d4d6b6a6 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:56.374876 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bbc11-4570-47a3-a06a-e776d4d6b6a6 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:56.375171 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bbc11-4570-47a3-a06a-e776d4d6b6a6 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:56.376569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1234/2454] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 17:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:40:56.389103 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf66d89-52b4-4faf-bfae-c8ac382b5d21 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:56.389806 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf66d89-52b4-4faf-bfae-c8ac382b5d21 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:56.390333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf66d89-52b4-4faf-bfae-c8ac382b5d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:56.421207 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:56.430203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-1bf66d89-52b4-4faf-bfae-c8ac382b5d21 None None] MFA Rules not processed for user `689a31e26c6b454a91c3fd79678b4d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:56.432192 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-1bf66d89-52b4-4faf-bfae-c8ac382b5d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:56.522557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb42f34-1645-48e9-9660-3dbf3ed00b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:56.527503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb42f34-1645-48e9-9660-3dbf3ed00b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:56.528041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb42f34-1645-48e9-9660-3dbf3ed00b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:56.529217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb42f34-1645-48e9-9660-3dbf3ed00b6a admin admin] PATH_INFO: `/v3/projects/4904f0e2f64e4bef989f441380e843b5` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:56.533795 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb42f34-1645-48e9-9660-3dbf3ed00b6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4904f0e2f64e4bef989f441380e843b5)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:56.537269 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb42f34-1645-48e9-9660-3dbf3ed00b6a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:56.574984 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1235/2455] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 17:40:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:56.591318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf66d89-52b4-4faf-bfae-c8ac382b5d21 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:56.593051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-1bf66d89-52b4-4faf-bfae-c8ac382b5d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:56.633946 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2cb42f34-1645-48e9-9660-3dbf3ed00b6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4904f0e2f64e4bef989f441380e843b5', 'request_id': 'req-2cb42f34-1645-48e9-9660-3dbf3ed00b6a'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:56.647738 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2cb42f34-1645-48e9-9660-3dbf3ed00b6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4904f0e2f64e4bef989f441380e843b5', 'request_id': 'req-2cb42f34-1645-48e9-9660-3dbf3ed00b6a'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:56.662199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2cb42f34-1645-48e9-9660-3dbf3ed00b6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4904f0e2f64e4bef989f441380e843b5', 'request_id': 'req-2cb42f34-1645-48e9-9660-3dbf3ed00b6a'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:56.691939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2cb42f34-1645-48e9-9660-3dbf3ed00b6a admin admin] The token cache is being invalidate because project 4904f0e2f64e4bef989f441380e843b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:40:56.692462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2cb42f34-1645-48e9-9660-3dbf3ed00b6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:56.693270 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-2cb42f34-1645-48e9-9660-3dbf3ed00b6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:40:56.700050 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1221/2456] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:40:56 2022] DELETE /identity/v3/projects/4904f0e2f64e4bef989f441380e843b5 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:56.700370 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a3126-297c-4366-95f4-a4a4b5413442 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:56.700804 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a3126-297c-4366-95f4-a4a4b5413442 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:56.701217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a3126-297c-4366-95f4-a4a4b5413442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:56.732643 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf66d89-52b4-4faf-bfae-c8ac382b5d21 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:56.744613 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-397a3126-297c-4366-95f4-a4a4b5413442 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:56.745858 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-397a3126-297c-4366-95f4-a4a4b5413442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:56.767051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:56.891309 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c69c4-6553-4140-b0ef-300239220383 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327b347d6295422c99ce7b44eab00467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:56.892643 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c69c4-6553-4140-b0ef-300239220383 service glance] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:56.895178 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c69c4-6553-4140-b0ef-300239220383 service glance] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:56.895887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c69c4-6553-4140-b0ef-300239220383 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:56.907948 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-cb7c69c4-6553-4140-b0ef-300239220383 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:56.924607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1222/2457] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:40:56.933542 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1fdc2-b26f-4a80-8409-ed53ef0e98f8 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:56.933915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1fdc2-b26f-4a80-8409-ed53ef0e98f8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:56.934265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1fdc2-b26f-4a80-8409-ed53ef0e98f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:56.987375 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-89e1fdc2-b26f-4a80-8409-ed53ef0e98f8 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:56.989995 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-89e1fdc2-b26f-4a80-8409-ed53ef0e98f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:57.006777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1223/2458] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:40:57.008151 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1fdc2-b26f-4a80-8409-ed53ef0e98f8 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:57.020347 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1fdc2-b26f-4a80-8409-ed53ef0e98f8 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:57.028615 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2e2f9-84a6-49db-863b-a363e0b57c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:57.030272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2e2f9-84a6-49db-863b-a363e0b57c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:57.030272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2e2f9-84a6-49db-863b-a363e0b57c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:57.030272 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2e2f9-84a6-49db-863b-a363e0b57c2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:57.032009 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d2e2f9-84a6-49db-863b-a363e0b57c2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:40:57.032428 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d2e2f9-84a6-49db-863b-a363e0b57c2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:57.035320 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d2e2f9-84a6-49db-863b-a363e0b57c2a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:57.077722 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1224/2459] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:40:57.078975 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2e2f9-84a6-49db-863b-a363e0b57c2a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:57.092214 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2e2f9-84a6-49db-863b-a363e0b57c2a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:57.096970 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c69c4-6553-4140-b0ef-300239220383 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:57.100271 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c69c4-6553-4140-b0ef-300239220383 service glance] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:57.105414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd7269-e50a-4dde-9c3f-7b84d445334e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:57.106211 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd7269-e50a-4dde-9c3f-7b84d445334e admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:57.106602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd7269-e50a-4dde-9c3f-7b84d445334e admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:57.107085 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd7269-e50a-4dde-9c3f-7b84d445334e admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:57.109706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd7269-e50a-4dde-9c3f-7b84d445334e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:57.115729 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1236/2460] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:40:57.115873 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd7269-e50a-4dde-9c3f-7b84d445334e admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:57.191655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1225/2461] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:40:57 2022] POST /identity/v3/projects => generated 363 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:57.202147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c69c4-6553-4140-b0ef-300239220383 service glance] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:57.223983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c69c4-6553-4140-b0ef-300239220383 service glance] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:57.272561 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48ca55-26c7-475d-b66c-3e6067ddeab4 None None] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:57.273115 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48ca55-26c7-475d-b66c-3e6067ddeab4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:57.273594 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48ca55-26c7-475d-b66c-3e6067ddeab4 None None] PATH_INFO: `/` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:57.275430 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1226/2462] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 17:40:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:40:57.293469 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf8c17-d60f-4d3a-9848-11ebbad21a05 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:57.293469 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf8c17-d60f-4d3a-9848-11ebbad21a05 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:57.294497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf8c17-d60f-4d3a-9848-11ebbad21a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:57.341780 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e7cf8c17-d60f-4d3a-9848-11ebbad21a05 None None] MFA Rules not processed for user `a303489493ce4cd2b62e876fbc8a7404`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:57.345763 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e7cf8c17-d60f-4d3a-9848-11ebbad21a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:57.356128 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c4a205-1be9-4149-9c9b-54607725b7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:57.357134 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4a205-1be9-4149-9c9b-54607725b7ec admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:57.358150 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4a205-1be9-4149-9c9b-54607725b7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:57.359246 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4a205-1be9-4149-9c9b-54607725b7ec admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:57.360170 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c4a205-1be9-4149-9c9b-54607725b7ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:57.362386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c4a205-1be9-4149-9c9b-54607725b7ec admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:57.398639 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:57.405217 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1237/2463] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:40:57 2022] POST /identity/v3/users => generated 282 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:57.410682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c4a205-1be9-4149-9c9b-54607725b7ec admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:57.432437 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c4a205-1be9-4149-9c9b-54607725b7ec admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:57.440587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4f1a1-e746-4e76-b6c4-7d92d7af0784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:57.442379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4f1a1-e746-4e76-b6c4-7d92d7af0784 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:57.442865 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4f1a1-e746-4e76-b6c4-7d92d7af0784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:57.443226 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4f1a1-e746-4e76-b6c4-7d92d7af0784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:57.444147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a4f1a1-e746-4e76-b6c4-7d92d7af0784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:57.446847 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a4f1a1-e746-4e76-b6c4-7d92d7af0784 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:57.475139 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1238/2464] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:40:57.476326 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4f1a1-e746-4e76-b6c4-7d92d7af0784 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:57.480736 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1227/2465] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 17:40:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:40:57.491247 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4f1a1-e746-4e76-b6c4-7d92d7af0784 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:57.496555 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c6cb5-fe1e-449d-812c-4d85a4c1939c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:57.497468 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c6cb5-fe1e-449d-812c-4d85a4c1939c admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:57.497881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c6cb5-fe1e-449d-812c-4d85a4c1939c admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:57.498185 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c6cb5-fe1e-449d-812c-4d85a4c1939c admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:57.499090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c6cb5-fe1e-449d-812c-4d85a4c1939c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:57.499206 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf8c17-d60f-4d3a-9848-11ebbad21a05 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:57.500765 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c6cb5-fe1e-449d-812c-4d85a4c1939c admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:57.501220 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e7cf8c17-d60f-4d3a-9848-11ebbad21a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:57.520553 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf8c17-d60f-4d3a-9848-11ebbad21a05 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:57.524753 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1239/2466] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:40:57.526151 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c6cb5-fe1e-449d-812c-4d85a4c1939c admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:57.531871 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee0c19-9e73-4462-be29-783618814a9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:57.532555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee0c19-9e73-4462-be29-783618814a9c service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:57.532850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee0c19-9e73-4462-be29-783618814a9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:57.533132 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee0c19-9e73-4462-be29-783618814a9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:57.541800 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c6cb5-fe1e-449d-812c-4d85a4c1939c admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:57.552732 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee0c19-9e73-4462-be29-783618814a9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:57.555117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee0c19-9e73-4462-be29-783618814a9c service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:57.555928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a717a3-c3c1-4e24-aaf9-ec12c6df0cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:57.557175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a717a3-c3c1-4e24-aaf9-ec12c6df0cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:57.557461 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a717a3-c3c1-4e24-aaf9-ec12c6df0cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:57.557748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a717a3-c3c1-4e24-aaf9-ec12c6df0cc3 admin admin] PATH_INFO: `/v3/projects/5d8006bcae0b4b0085d2cb6ee492636d/users/934285e99715446ea031acc56b52e14e/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:57.575299 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1228/2467] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:40:57.588635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a717a3-c3c1-4e24-aaf9-ec12c6df0cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d8006bcae0b4b0085d2cb6ee492636d, user_id=934285e99715446ea031acc56b52e14e, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:57.591386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a717a3-c3c1-4e24-aaf9-ec12c6df0cc3 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:57.613398 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:57.631383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1240/2468] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:40:57 2022] PUT /identity/v3/projects/5d8006bcae0b4b0085d2cb6ee492636d/users/934285e99715446ea031acc56b52e14e/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:57.645875 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bdf83-ad0d-44e9-8a71-a19d3c9c680a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:57.646837 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bdf83-ad0d-44e9-8a71-a19d3c9c680a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:57.647133 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bdf83-ad0d-44e9-8a71-a19d3c9c680a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:57.705758 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-bd5bdf83-ad0d-44e9-8a71-a19d3c9c680a None None] MFA Rules not processed for user `934285e99715446ea031acc56b52e14e`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:57.708029 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-bd5bdf83-ad0d-44e9-8a71-a19d3c9c680a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:57.775201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf5d58-e772-4f69-8068-bc9b57b478f3 None None] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:57.775502 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf5d58-e772-4f69-8068-bc9b57b478f3 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:57.775795 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf5d58-e772-4f69-8068-bc9b57b478f3 None None] PATH_INFO: `/` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:57.777415 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1241/2469] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 17:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:40:57.786671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e040b6-89d5-47fb-a94f-3d742674bf77 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:57.787537 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e040b6-89d5-47fb-a94f-3d742674bf77 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:57.788065 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e040b6-89d5-47fb-a94f-3d742674bf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:57.814588 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-08e040b6-89d5-47fb-a94f-3d742674bf77 None None] MFA Rules not processed for user `689a31e26c6b454a91c3fd79678b4d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:57.817036 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-08e040b6-89d5-47fb-a94f-3d742674bf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:57.852453 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1229/2470] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:57 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:57.853758 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bdf83-ad0d-44e9-8a71-a19d3c9c680a None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:58.038270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1242/2471] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 17:40:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:40:58.039575 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e040b6-89d5-47fb-a94f-3d742674bf77 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:58.061828 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bdf83-ad0d-44e9-8a71-a19d3c9c680a None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:58.075145 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370fa42-2433-4e6a-a985-60a260522bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:58.079278 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370fa42-2433-4e6a-a985-60a260522bbf admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:58.080637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370fa42-2433-4e6a-a985-60a260522bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:58.081013 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370fa42-2433-4e6a-a985-60a260522bbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:58.082580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2370fa42-2433-4e6a-a985-60a260522bbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:58.088796 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2370fa42-2433-4e6a-a985-60a260522bbf admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:58.175421 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1230/2472] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:40:57 2022] POST /identity/v3/projects => generated 365 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:58.178026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370fa42-2433-4e6a-a985-60a260522bbf admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:58.205937 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e040b6-89d5-47fb-a94f-3d742674bf77 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:58.342848 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:58.385318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c8ffe-7df4-44cb-b3ed-5b1f60bb4b16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:58.385318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c8ffe-7df4-44cb-b3ed-5b1f60bb4b16 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:58.385318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c8ffe-7df4-44cb-b3ed-5b1f60bb4b16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:58.385318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c8ffe-7df4-44cb-b3ed-5b1f60bb4b16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:58.427550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370fa42-2433-4e6a-a985-60a260522bbf admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:58.442863 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6593f-b03a-42ae-9038-d2b24848238a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:58.446687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6593f-b03a-42ae-9038-d2b24848238a admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:58.447219 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6593f-b03a-42ae-9038-d2b24848238a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:58.447695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6593f-b03a-42ae-9038-d2b24848238a admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:58.448666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a6593f-b03a-42ae-9038-d2b24848238a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:58.455855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a6593f-b03a-42ae-9038-d2b24848238a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:58.529826 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:58.547800 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1231/2473] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:40:58 2022] POST /identity/v3/users => generated 289 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:40:58.552956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6593f-b03a-42ae-9038-d2b24848238a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:58.578867 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6593f-b03a-42ae-9038-d2b24848238a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:58.583901 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c8ffe-7df4-44cb-b3ed-5b1f60bb4b16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:58.585527 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c8ffe-7df4-44cb-b3ed-5b1f60bb4b16 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:58.589088 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63a4aa-1d73-465d-b709-5d02493535ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:58.590241 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63a4aa-1d73-465d-b709-5d02493535ec admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:58.590731 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63a4aa-1d73-465d-b709-5d02493535ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:58.591544 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63a4aa-1d73-465d-b709-5d02493535ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:58.592586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63a4aa-1d73-465d-b709-5d02493535ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:58.594856 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63a4aa-1d73-465d-b709-5d02493535ec admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:58.617317 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1232/2474] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:40:58.624936 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63a4aa-1d73-465d-b709-5d02493535ec admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:58.636435 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1243/2475] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:40:58.694000 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63a4aa-1d73-465d-b709-5d02493535ec admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:58.718656 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deda660-bbe3-40d9-91be-ecd134a1ccb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:58.719880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deda660-bbe3-40d9-91be-ecd134a1ccb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:58.720015 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deda660-bbe3-40d9-91be-ecd134a1ccb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:58.720217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deda660-bbe3-40d9-91be-ecd134a1ccb4 admin admin] PATH_INFO: `/v3/projects/31c47f78ea6846d78156cfa85a23cb19/users/eda2b4c5d95b47c2a49659bff70c9718/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:58.742911 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deda660-bbe3-40d9-91be-ecd134a1ccb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31c47f78ea6846d78156cfa85a23cb19, user_id=eda2b4c5d95b47c2a49659bff70c9718, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:58.745684 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deda660-bbe3-40d9-91be-ecd134a1ccb4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:58.775318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1233/2476] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:40:58 2022] PUT /identity/v3/projects/31c47f78ea6846d78156cfa85a23cb19/users/eda2b4c5d95b47c2a49659bff70c9718/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:58.779742 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c8ffe-7df4-44cb-b3ed-5b1f60bb4b16 service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:58.812677 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c8ffe-7df4-44cb-b3ed-5b1f60bb4b16 service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:58.965063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9867b-0c45-4e97-997a-49340e9bbdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:58.965808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9867b-0c45-4e97-997a-49340e9bbdb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:58.966098 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9867b-0c45-4e97-997a-49340e9bbdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:58.966533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9867b-0c45-4e97-997a-49340e9bbdb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:58.967433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9867b-0c45-4e97-997a-49340e9bbdb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:58.977744 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9867b-0c45-4e97-997a-49340e9bbdb9 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:59.004640 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1244/2477] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:40:59.006434 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deda660-bbe3-40d9-91be-ecd134a1ccb4 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:59.022423 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deda660-bbe3-40d9-91be-ecd134a1ccb4 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:59.031598 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bae6f6-f582-4fa5-b5f0-9f6fdff07bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:59.032537 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bae6f6-f582-4fa5-b5f0-9f6fdff07bef admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:59.032999 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bae6f6-f582-4fa5-b5f0-9f6fdff07bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:59.034117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bae6f6-f582-4fa5-b5f0-9f6fdff07bef admin admin] PATH_INFO: `/v3/projects/31c47f78ea6846d78156cfa85a23cb19/users/eda2b4c5d95b47c2a49659bff70c9718/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:59.038097 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bae6f6-f582-4fa5-b5f0-9f6fdff07bef admin admin] RBAC: Authorizing `identity:create_grant(project_id=31c47f78ea6846d78156cfa85a23cb19, user_id=eda2b4c5d95b47c2a49659bff70c9718, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:59.041928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bae6f6-f582-4fa5-b5f0-9f6fdff07bef admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:59.080045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1234/2478] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:40:59 2022] PUT /identity/v3/projects/31c47f78ea6846d78156cfa85a23cb19/users/eda2b4c5d95b47c2a49659bff70c9718/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:59.082150 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47ff2e-c36e-4e1f-b85b-dab2fda4a96f None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:59.082498 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47ff2e-c36e-4e1f-b85b-dab2fda4a96f None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:59.083071 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47ff2e-c36e-4e1f-b85b-dab2fda4a96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:59.133168 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-6c47ff2e-c36e-4e1f-b85b-dab2fda4a96f None None] MFA Rules not processed for user `eda2b4c5d95b47c2a49659bff70c9718`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:40:59.137345 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-6c47ff2e-c36e-4e1f-b85b-dab2fda4a96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:59.285382 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1245/2479] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:40:59 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:40:59.309309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bae6f6-f582-4fa5-b5f0-9f6fdff07bef admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:40:59.311432 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-71bae6f6-f582-4fa5-b5f0-9f6fdff07bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:40:59.424314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bae6f6-f582-4fa5-b5f0-9f6fdff07bef admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:40:59.431314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fad3d4-19c9-4191-b845-87079cfbc2de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:40:59.432834 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fad3d4-19c9-4191-b845-87079cfbc2de service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:40:59.432834 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fad3d4-19c9-4191-b845-87079cfbc2de service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:40:59.432834 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fad3d4-19c9-4191-b845-87079cfbc2de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:40:59.447830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fad3d4-19c9-4191-b845-87079cfbc2de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:40:59.449440 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fad3d4-19c9-4191-b845-87079cfbc2de service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:40:59.469566 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1235/2480] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:40:59 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:00.443692 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47ff2e-c36e-4e1f-b85b-dab2fda4a96f None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:00.544562 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47ff2e-c36e-4e1f-b85b-dab2fda4a96f None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:00.549802 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78520a69-bdc2-4a8a-9d1e-00358dcfebfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:00.551338 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78520a69-bdc2-4a8a-9d1e-00358dcfebfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:00.551651 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78520a69-bdc2-4a8a-9d1e-00358dcfebfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:00.551909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78520a69-bdc2-4a8a-9d1e-00358dcfebfc admin admin] PATH_INFO: `/v3/users/934285e99715446ea031acc56b52e14e` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:00.553305 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78520a69-bdc2-4a8a-9d1e-00358dcfebfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=934285e99715446ea031acc56b52e14e)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:00.555189 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78520a69-bdc2-4a8a-9d1e-00358dcfebfc admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:00.567742 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:00.634827 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-78520a69-bdc2-4a8a-9d1e-00358dcfebfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '934285e99715446ea031acc56b52e14e'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:00.649339 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-78520a69-bdc2-4a8a-9d1e-00358dcfebfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '934285e99715446ea031acc56b52e14e'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:00.650651 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-78520a69-bdc2-4a8a-9d1e-00358dcfebfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '934285e99715446ea031acc56b52e14e'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:00.660163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-78520a69-bdc2-4a8a-9d1e-00358dcfebfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '934285e99715446ea031acc56b52e14e'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:00.684752 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-78520a69-bdc2-4a8a-9d1e-00358dcfebfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '934285e99715446ea031acc56b52e14e'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:00.693299 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1246/2481] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:41:00 2022] DELETE /identity/v3/users/934285e99715446ea031acc56b52e14e => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:00.698821 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40514a-2f24-4065-b173-1a9923382396 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:00.699834 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40514a-2f24-4065-b173-1a9923382396 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:00.700139 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40514a-2f24-4065-b173-1a9923382396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:00.721579 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-3f40514a-2f24-4065-b173-1a9923382396 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:00.723703 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3f40514a-2f24-4065-b173-1a9923382396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:00.801725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1236/2482] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:00.812314 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78520a69-bdc2-4a8a-9d1e-00358dcfebfc admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:00.814487 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-78520a69-bdc2-4a8a-9d1e-00358dcfebfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:00.888727 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78520a69-bdc2-4a8a-9d1e-00358dcfebfc admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:00.895686 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9e3b7-8cdc-4be1-9a07-4753fd64d5d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:00.895686 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9e3b7-8cdc-4be1-9a07-4753fd64d5d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:00.895686 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9e3b7-8cdc-4be1-9a07-4753fd64d5d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:00.895686 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9e3b7-8cdc-4be1-9a07-4753fd64d5d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:00.905911 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9e3b7-8cdc-4be1-9a07-4753fd64d5d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:00.907408 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9e3b7-8cdc-4be1-9a07-4753fd64d5d9 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:00.919914 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1247/2483] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:01.375510 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f40514a-2f24-4065-b173-1a9923382396 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:01.377197 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3f40514a-2f24-4065-b173-1a9923382396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:01.397676 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f40514a-2f24-4065-b173-1a9923382396 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:01.403721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385c44b-9ab3-4fd7-a453-15144b4846a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:01.405979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385c44b-9ab3-4fd7-a453-15144b4846a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:01.406198 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385c44b-9ab3-4fd7-a453-15144b4846a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:01.420037 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385c44b-9ab3-4fd7-a453-15144b4846a0 admin admin] PATH_INFO: `/v3/projects/5d8006bcae0b4b0085d2cb6ee492636d` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:01.427846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385c44b-9ab3-4fd7-a453-15144b4846a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d8006bcae0b4b0085d2cb6ee492636d)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:01.427846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385c44b-9ab3-4fd7-a453-15144b4846a0 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:01.439847 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9e3b7-8cdc-4be1-9a07-4753fd64d5d9 service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:01.451853 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9e3b7-8cdc-4be1-9a07-4753fd64d5d9 service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:01.459536 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c411826-898c-4450-9cef-25d2c1afe774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:01.460203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c411826-898c-4450-9cef-25d2c1afe774 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:01.460671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c411826-898c-4450-9cef-25d2c1afe774 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:01.461084 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c411826-898c-4450-9cef-25d2c1afe774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:01.463112 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0c411826-898c-4450-9cef-25d2c1afe774 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:01.463696 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:01.489942 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c411826-898c-4450-9cef-25d2c1afe774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:01.491712 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c411826-898c-4450-9cef-25d2c1afe774 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:01.508135 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1248/2484] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:01.553813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5385c44b-9ab3-4fd7-a453-15144b4846a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d8006bcae0b4b0085d2cb6ee492636d', 'request_id': 'req-5385c44b-9ab3-4fd7-a453-15144b4846a0'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:01.569374 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5385c44b-9ab3-4fd7-a453-15144b4846a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d8006bcae0b4b0085d2cb6ee492636d', 'request_id': 'req-5385c44b-9ab3-4fd7-a453-15144b4846a0'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:01.586789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5385c44b-9ab3-4fd7-a453-15144b4846a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d8006bcae0b4b0085d2cb6ee492636d', 'request_id': 'req-5385c44b-9ab3-4fd7-a453-15144b4846a0'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:01.605036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5385c44b-9ab3-4fd7-a453-15144b4846a0 admin admin] The token cache is being invalidate because project 5d8006bcae0b4b0085d2cb6ee492636d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:41:01.605432 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5385c44b-9ab3-4fd7-a453-15144b4846a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:01.606206 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5385c44b-9ab3-4fd7-a453-15144b4846a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:01.609020 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1237/2485] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:41:01 2022] DELETE /identity/v3/projects/5d8006bcae0b4b0085d2cb6ee492636d => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:01.613256 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c411826-898c-4450-9cef-25d2c1afe774 service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:01.632863 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c411826-898c-4450-9cef-25d2c1afe774 service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:01.752244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59925843-cd3c-45fc-9235-e915bd922356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:01.753767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59925843-cd3c-45fc-9235-e915bd922356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:01.754334 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59925843-cd3c-45fc-9235-e915bd922356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:01.754557 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59925843-cd3c-45fc-9235-e915bd922356 admin admin] PATH_INFO: `/v3/users/eda2b4c5d95b47c2a49659bff70c9718` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:01.755839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59925843-cd3c-45fc-9235-e915bd922356 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda2b4c5d95b47c2a49659bff70c9718)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:01.757915 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59925843-cd3c-45fc-9235-e915bd922356 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:01.816852 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:01.879575 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-59925843-cd3c-45fc-9235-e915bd922356 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda2b4c5d95b47c2a49659bff70c9718'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:01.897527 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-59925843-cd3c-45fc-9235-e915bd922356 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda2b4c5d95b47c2a49659bff70c9718'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:01.898576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-59925843-cd3c-45fc-9235-e915bd922356 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda2b4c5d95b47c2a49659bff70c9718'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:01.916194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-59925843-cd3c-45fc-9235-e915bd922356 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda2b4c5d95b47c2a49659bff70c9718'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:01.939835 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-59925843-cd3c-45fc-9235-e915bd922356 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda2b4c5d95b47c2a49659bff70c9718'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:01.949366 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1249/2486] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:41:01 2022] DELETE /identity/v3/users/eda2b4c5d95b47c2a49659bff70c9718 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:01.980473 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c068c67-d5e5-4f8c-a8a1-d16dd3954a47 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:01.981045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c068c67-d5e5-4f8c-a8a1-d16dd3954a47 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:01.981045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c068c67-d5e5-4f8c-a8a1-d16dd3954a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:02.013710 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-2c068c67-d5e5-4f8c-a8a1-d16dd3954a47 None None] MFA Rules not processed for user `cfc2f31f654e486d8505d80aad17d17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:02.015366 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2c068c67-d5e5-4f8c-a8a1-d16dd3954a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:02.101521 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1238/2487] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 17:41:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:02.115284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59925843-cd3c-45fc-9235-e915bd922356 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:02.117193 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-59925843-cd3c-45fc-9235-e915bd922356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:02.279563 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59925843-cd3c-45fc-9235-e915bd922356 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:02.299644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f15e59-31f3-4d18-ade7-ffbaebe80faf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a37d1201d54afdad9b228ef4f2d787', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:02.300842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f15e59-31f3-4d18-ade7-ffbaebe80faf service swift] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:02.301137 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f15e59-31f3-4d18-ade7-ffbaebe80faf service swift] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:02.301489 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f15e59-31f3-4d18-ade7-ffbaebe80faf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:02.322962 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f15e59-31f3-4d18-ade7-ffbaebe80faf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:02.325560 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f15e59-31f3-4d18-ade7-ffbaebe80faf service swift] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:02.343454 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1250/2488] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 17:41:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:41:02.846074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c068c67-d5e5-4f8c-a8a1-d16dd3954a47 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:02.849333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2c068c67-d5e5-4f8c-a8a1-d16dd3954a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:03.273021 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c068c67-d5e5-4f8c-a8a1-d16dd3954a47 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:03.283568 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6d7d2-6d3c-4bea-89cb-4587dad0743e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:03.286309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6d7d2-6d3c-4bea-89cb-4587dad0743e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:03.289404 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6d7d2-6d3c-4bea-89cb-4587dad0743e admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:03.290606 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6d7d2-6d3c-4bea-89cb-4587dad0743e admin admin] PATH_INFO: `/v3/projects/31c47f78ea6846d78156cfa85a23cb19` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:03.293402 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd6d7d2-6d3c-4bea-89cb-4587dad0743e admin admin] RBAC: Authorizing `identity:delete_project(project_id=31c47f78ea6846d78156cfa85a23cb19)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:03.298082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd6d7d2-6d3c-4bea-89cb-4587dad0743e admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:03.435522 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0bd6d7d2-6d3c-4bea-89cb-4587dad0743e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31c47f78ea6846d78156cfa85a23cb19', 'request_id': 'req-0bd6d7d2-6d3c-4bea-89cb-4587dad0743e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:03.439546 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:03.478323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0bd6d7d2-6d3c-4bea-89cb-4587dad0743e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31c47f78ea6846d78156cfa85a23cb19', 'request_id': 'req-0bd6d7d2-6d3c-4bea-89cb-4587dad0743e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:03.500028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0bd6d7d2-6d3c-4bea-89cb-4587dad0743e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31c47f78ea6846d78156cfa85a23cb19', 'request_id': 'req-0bd6d7d2-6d3c-4bea-89cb-4587dad0743e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:03.540643 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0bd6d7d2-6d3c-4bea-89cb-4587dad0743e admin admin] The token cache is being invalidate because project 31c47f78ea6846d78156cfa85a23cb19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:41:03.542233 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0bd6d7d2-6d3c-4bea-89cb-4587dad0743e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:03.542999 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0bd6d7d2-6d3c-4bea-89cb-4587dad0743e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:03.549416 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1239/2489] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:41:02 2022] DELETE /identity/v3/projects/31c47f78ea6846d78156cfa85a23cb19 => generated 0 bytes in 705 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:41:03.554461 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e6cb1-d544-4d32-9df2-046639ea74f1 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:03.555055 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e6cb1-d544-4d32-9df2-046639ea74f1 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:03.555458 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e6cb1-d544-4d32-9df2-046639ea74f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:03.589723 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-828e6cb1-d544-4d32-9df2-046639ea74f1 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:03.592729 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-828e6cb1-d544-4d32-9df2-046639ea74f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:03.734125 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b608e26-0671-4fc2-9132-0167aaaa66cc None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:03.734593 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b608e26-0671-4fc2-9132-0167aaaa66cc None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:03.734721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b608e26-0671-4fc2-9132-0167aaaa66cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:03.768533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1251/2490] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:03.781130 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a39fa-8da3-483c-bb47-086c384c0712 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:03.781822 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a39fa-8da3-483c-bb47-086c384c0712 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:03.782128 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a39fa-8da3-483c-bb47-086c384c0712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:03.783562 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-5b608e26-0671-4fc2-9132-0167aaaa66cc None None] MFA Rules not processed for user `7a3db4c2153340eeaf907c2fe0919e41`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:03.785083 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5b608e26-0671-4fc2-9132-0167aaaa66cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:03.830408 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-b46a39fa-8da3-483c-bb47-086c384c0712 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:03.832671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b46a39fa-8da3-483c-bb47-086c384c0712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:03.866676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1252/2491] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:03.873193 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a39fa-8da3-483c-bb47-086c384c0712 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:03.898746 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a39fa-8da3-483c-bb47-086c384c0712 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:03.906048 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84351fe-436d-42ba-a428-0d785cfd3ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:03.907076 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84351fe-436d-42ba-a428-0d785cfd3ced admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:03.907953 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84351fe-436d-42ba-a428-0d785cfd3ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:03.908743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84351fe-436d-42ba-a428-0d785cfd3ced admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:03.914366 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84351fe-436d-42ba-a428-0d785cfd3ced admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:41:03.914770 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84351fe-436d-42ba-a428-0d785cfd3ced admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:03.916811 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84351fe-436d-42ba-a428-0d785cfd3ced admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:03.937991 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1253/2492] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:41:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:41:03.939196 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84351fe-436d-42ba-a428-0d785cfd3ced admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:03.953574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84351fe-436d-42ba-a428-0d785cfd3ced admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:03.962038 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df69ef3-1de6-4601-89d8-236ff436fc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:03.962737 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df69ef3-1de6-4601-89d8-236ff436fc59 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:03.963084 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df69ef3-1de6-4601-89d8-236ff436fc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:03.963374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df69ef3-1de6-4601-89d8-236ff436fc59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:03.964089 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df69ef3-1de6-4601-89d8-236ff436fc59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:03.966176 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df69ef3-1de6-4601-89d8-236ff436fc59 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:03.974616 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:03.974947 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1240/2493] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:41:03.989704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b608e26-0671-4fc2-9132-0167aaaa66cc None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:03.991250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5b608e26-0671-4fc2-9132-0167aaaa66cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:04.068273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1254/2494] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:41:03 2022] POST /identity/v3/projects => generated 381 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:04.073887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df69ef3-1de6-4601-89d8-236ff436fc59 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:04.193783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b608e26-0671-4fc2-9132-0167aaaa66cc None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:04.374449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df69ef3-1de6-4601-89d8-236ff436fc59 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:04.377216 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b707d-c0e0-4b7a-a9d5-621c38b9ff93 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327b347d6295422c99ce7b44eab00467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:04.410637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b707d-c0e0-4b7a-a9d5-621c38b9ff93 service glance] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:04.410637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b707d-c0e0-4b7a-a9d5-621c38b9ff93 service glance] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:04.410637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b707d-c0e0-4b7a-a9d5-621c38b9ff93 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:04.412132 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9abaa5-c20c-43f7-b40e-8135fe0132c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:04.413049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9abaa5-c20c-43f7-b40e-8135fe0132c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:04.413402 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9abaa5-c20c-43f7-b40e-8135fe0132c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:04.413718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9abaa5-c20c-43f7-b40e-8135fe0132c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:04.414421 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9abaa5-c20c-43f7-b40e-8135fe0132c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:04.416839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9abaa5-c20c-43f7-b40e-8135fe0132c2 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:04.420790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b64b707d-c0e0-4b7a-a9d5-621c38b9ff93 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:04.506568 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:04.545441 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b707d-c0e0-4b7a-a9d5-621c38b9ff93 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:04.549068 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b707d-c0e0-4b7a-a9d5-621c38b9ff93 service glance] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:04.579141 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1255/2495] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:41:04 2022] POST /identity/v3/users => generated 291 bytes in 506 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:04.584130 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9abaa5-c20c-43f7-b40e-8135fe0132c2 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:04.616935 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1241/2496] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:04.635735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9abaa5-c20c-43f7-b40e-8135fe0132c2 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:04.668737 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc01fd6-784f-4e58-a276-a05d7d04a705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:04.670443 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc01fd6-784f-4e58-a276-a05d7d04a705 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:04.671826 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc01fd6-784f-4e58-a276-a05d7d04a705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:04.675308 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc01fd6-784f-4e58-a276-a05d7d04a705 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:04.677953 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc01fd6-784f-4e58-a276-a05d7d04a705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:04.684699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc01fd6-784f-4e58-a276-a05d7d04a705 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:04.734514 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1256/2497] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:04.750135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b707d-c0e0-4b7a-a9d5-621c38b9ff93 service glance] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:04.783288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b707d-c0e0-4b7a-a9d5-621c38b9ff93 service glance] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:04.807179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c1aa1-99c7-4e30-a474-e46c067d7cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:04.808517 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c1aa1-99c7-4e30-a474-e46c067d7cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:04.809397 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c1aa1-99c7-4e30-a474-e46c067d7cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:04.810034 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c1aa1-99c7-4e30-a474-e46c067d7cf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:04.811021 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c1aa1-99c7-4e30-a474-e46c067d7cf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:04.813441 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c1aa1-99c7-4e30-a474-e46c067d7cf3 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:04.864111 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1242/2498] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:04.867117 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc01fd6-784f-4e58-a276-a05d7d04a705 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:04.898234 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc01fd6-784f-4e58-a276-a05d7d04a705 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:04.919912 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302b497-19e2-4d2e-ad6e-5f05766512ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:04.922171 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302b497-19e2-4d2e-ad6e-5f05766512ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:04.922661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302b497-19e2-4d2e-ad6e-5f05766512ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:04.922958 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302b497-19e2-4d2e-ad6e-5f05766512ac admin admin] PATH_INFO: `/v3/projects/f64c728f2f174894a788ceeea669aa8d/users/523d2917d8214af0b496ae813df81090/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:04.961533 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89989712-6807-4565-9e22-c48a83e81eaf None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:04.961967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89989712-6807-4565-9e22-c48a83e81eaf None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:04.962536 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89989712-6807-4565-9e22-c48a83e81eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:04.986354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302b497-19e2-4d2e-ad6e-5f05766512ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f64c728f2f174894a788ceeea669aa8d, user_id=523d2917d8214af0b496ae813df81090, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:05.000408 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302b497-19e2-4d2e-ad6e-5f05766512ac admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:05.022214 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-89989712-6807-4565-9e22-c48a83e81eaf None None] MFA Rules not processed for user `cfc2f31f654e486d8505d80aad17d17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:05.027629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-89989712-6807-4565-9e22-c48a83e81eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:05.042091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1257/2499] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:04 2022] PUT /identity/v3/projects/f64c728f2f174894a788ceeea669aa8d/users/523d2917d8214af0b496ae813df81090/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:05.070000 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b6b75-22f9-409d-94c2-ac1ce9e9d24d None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:05.070832 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b6b75-22f9-409d-94c2-ac1ce9e9d24d None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:05.071831 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b6b75-22f9-409d-94c2-ac1ce9e9d24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:05.130086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-963b6b75-22f9-409d-94c2-ac1ce9e9d24d None None] MFA Rules not processed for user `523d2917d8214af0b496ae813df81090`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:05.131758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-963b6b75-22f9-409d-94c2-ac1ce9e9d24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:05.279291 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1243/2500] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 17:41:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:05.320631 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89989712-6807-4565-9e22-c48a83e81eaf None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:05.323336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-89989712-6807-4565-9e22-c48a83e81eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:05.385460 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1258/2501] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:41:05.386649 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b6b75-22f9-409d-94c2-ac1ce9e9d24d None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:05.482130 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:05.603176 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b6b75-22f9-409d-94c2-ac1ce9e9d24d None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:05.620961 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89989712-6807-4565-9e22-c48a83e81eaf None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:05.623252 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe51e9-9192-4ab0-bc4e-2b37708f8d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:05.624602 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe51e9-9192-4ab0-bc4e-2b37708f8d17 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:05.625175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe51e9-9192-4ab0-bc4e-2b37708f8d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:05.625798 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe51e9-9192-4ab0-bc4e-2b37708f8d17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:05.627424 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe51e9-9192-4ab0-bc4e-2b37708f8d17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:05.631378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe51e9-9192-4ab0-bc4e-2b37708f8d17 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:05.648643 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8042fc0-18ca-4e32-8f12-8a1876dc6443 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a37d1201d54afdad9b228ef4f2d787', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:05.649643 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8042fc0-18ca-4e32-8f12-8a1876dc6443 service swift] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:05.650125 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8042fc0-18ca-4e32-8f12-8a1876dc6443 service swift] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:05.650580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8042fc0-18ca-4e32-8f12-8a1876dc6443 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:05.693683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8042fc0-18ca-4e32-8f12-8a1876dc6443 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:05.700694 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8042fc0-18ca-4e32-8f12-8a1876dc6443 service swift] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:05.715881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1259/2502] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:41:05 2022] POST /identity/v3/projects => generated 381 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:05.721378 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe51e9-9192-4ab0-bc4e-2b37708f8d17 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:05.760744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1244/2503] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 17:41:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:41:05.869441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe51e9-9192-4ab0-bc4e-2b37708f8d17 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:05.879147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40badf61-21b2-4c16-8327-0e50ae7b35de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:05.880503 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40badf61-21b2-4c16-8327-0e50ae7b35de admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:05.881030 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40badf61-21b2-4c16-8327-0e50ae7b35de admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:05.881551 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40badf61-21b2-4c16-8327-0e50ae7b35de admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:05.883944 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40badf61-21b2-4c16-8327-0e50ae7b35de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:05.886806 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40badf61-21b2-4c16-8327-0e50ae7b35de admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:05.937655 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1260/2504] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:41:05 2022] POST /identity/v3/users => generated 297 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:05.943050 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8042fc0-18ca-4e32-8f12-8a1876dc6443 service swift] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:05.976284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8042fc0-18ca-4e32-8f12-8a1876dc6443 service swift] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:05.996520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad02b4b-cc39-41ad-b8f0-9d6e36f6096a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:05.997334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad02b4b-cc39-41ad-b8f0-9d6e36f6096a admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:05.997670 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad02b4b-cc39-41ad-b8f0-9d6e36f6096a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:05.997903 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad02b4b-cc39-41ad-b8f0-9d6e36f6096a admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:05.998744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad02b4b-cc39-41ad-b8f0-9d6e36f6096a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:06.000412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad02b4b-cc39-41ad-b8f0-9d6e36f6096a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:06.029734 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1245/2505] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:06.033982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40badf61-21b2-4c16-8327-0e50ae7b35de admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:06.063909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40badf61-21b2-4c16-8327-0e50ae7b35de admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:06.074242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc7c3b-38ad-40ed-8930-07081dd83f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:06.075344 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc7c3b-38ad-40ed-8930-07081dd83f04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:06.075832 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc7c3b-38ad-40ed-8930-07081dd83f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:06.075954 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc7c3b-38ad-40ed-8930-07081dd83f04 admin admin] PATH_INFO: `/v3/projects/ebca6edbf09c42debbe919838dab3ba4/users/2fb79c3e45ec44e39cac0b7523f3b45b/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:06.112073 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc7c3b-38ad-40ed-8930-07081dd83f04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebca6edbf09c42debbe919838dab3ba4, user_id=2fb79c3e45ec44e39cac0b7523f3b45b, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:06.119272 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc7c3b-38ad-40ed-8930-07081dd83f04 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:06.156164 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1261/2506] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:06 2022] PUT /identity/v3/projects/ebca6edbf09c42debbe919838dab3ba4/users/2fb79c3e45ec44e39cac0b7523f3b45b/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:41:06.156491 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad02b4b-cc39-41ad-b8f0-9d6e36f6096a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:06.179395 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad02b4b-cc39-41ad-b8f0-9d6e36f6096a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:06.275918 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:06.314022 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea35e2-ae2a-4782-911a-d9cad9847787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:06.314734 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea35e2-ae2a-4782-911a-d9cad9847787 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:06.315023 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea35e2-ae2a-4782-911a-d9cad9847787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:06.315334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea35e2-ae2a-4782-911a-d9cad9847787 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:06.316105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea35e2-ae2a-4782-911a-d9cad9847787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:06.318245 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea35e2-ae2a-4782-911a-d9cad9847787 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:06.353861 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc7c3b-38ad-40ed-8930-07081dd83f04 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:06.357798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1246/2507] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:41:06.372929 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc7c3b-38ad-40ed-8930-07081dd83f04 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:06.390443 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d33f6e-910f-4fe9-836a-3d09adfd40b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:06.393261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d33f6e-910f-4fe9-836a-3d09adfd40b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:06.397526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d33f6e-910f-4fe9-836a-3d09adfd40b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:06.398144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d33f6e-910f-4fe9-836a-3d09adfd40b1 admin admin] PATH_INFO: `/v3/projects/ebca6edbf09c42debbe919838dab3ba4/users/2fb79c3e45ec44e39cac0b7523f3b45b/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:06.411812 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d33f6e-910f-4fe9-836a-3d09adfd40b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebca6edbf09c42debbe919838dab3ba4, user_id=2fb79c3e45ec44e39cac0b7523f3b45b, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:06.415611 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d33f6e-910f-4fe9-836a-3d09adfd40b1 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:06.455292 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1262/2508] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:06 2022] PUT /identity/v3/projects/ebca6edbf09c42debbe919838dab3ba4/users/2fb79c3e45ec44e39cac0b7523f3b45b/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:06.458883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4ca91-5cb9-4759-8962-51c1cbe68f2a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:06.459323 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4ca91-5cb9-4759-8962-51c1cbe68f2a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:06.460785 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4ca91-5cb9-4759-8962-51c1cbe68f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:06.542653 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-36c4ca91-5cb9-4759-8962-51c1cbe68f2a None None] MFA Rules not processed for user `2fb79c3e45ec44e39cac0b7523f3b45b`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:06.544652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-36c4ca91-5cb9-4759-8962-51c1cbe68f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:06.810028 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1247/2509] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:06.825666 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d33f6e-910f-4fe9-836a-3d09adfd40b1 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:06.827827 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-c6d33f6e-910f-4fe9-836a-3d09adfd40b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:06.947952 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d33f6e-910f-4fe9-836a-3d09adfd40b1 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:06.954892 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196730e-d957-4120-8c37-dbba5565f162 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:06.955920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196730e-d957-4120-8c37-dbba5565f162 service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:06.956718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196730e-d957-4120-8c37-dbba5565f162 service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:06.957182 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196730e-d957-4120-8c37-dbba5565f162 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:06.971396 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196730e-d957-4120-8c37-dbba5565f162 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:06.973974 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196730e-d957-4120-8c37-dbba5565f162 service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:06.998285 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1263/2510] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:41:07.900974 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c4ca91-5cb9-4759-8962-51c1cbe68f2a None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:07.936267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c4ca91-5cb9-4759-8962-51c1cbe68f2a None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:07.978337 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d75ca8-3e6c-4217-b9eb-b8a7fe4557b4 tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fb79c3e45ec44e39cac0b7523f3b45b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebca6edbf09c42debbe919838dab3ba4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:07.990316 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d75ca8-3e6c-4217-b9eb-b8a7fe4557b4 tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:07.990889 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d75ca8-3e6c-4217-b9eb-b8a7fe4557b4 tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:07.991394 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d75ca8-3e6c-4217-b9eb-b8a7fe4557b4 tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:07.993178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d75ca8-3e6c-4217-b9eb-b8a7fe4557b4 tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:41:07.993345 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d75ca8-3e6c-4217-b9eb-b8a7fe4557b4 tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:07.996977 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d75ca8-3e6c-4217-b9eb-b8a7fe4557b4 tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:08.077937 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1248/2511] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:41:08.085701 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196730e-d957-4120-8c37-dbba5565f162 service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:08.130918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196730e-d957-4120-8c37-dbba5565f162 service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:08.157776 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a5a10-703e-476e-b294-497ca6c416f5 tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fb79c3e45ec44e39cac0b7523f3b45b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebca6edbf09c42debbe919838dab3ba4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:08.159908 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a5a10-703e-476e-b294-497ca6c416f5 tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:08.159908 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a5a10-703e-476e-b294-497ca6c416f5 tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:08.159908 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a5a10-703e-476e-b294-497ca6c416f5 tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:08.166844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a5a10-703e-476e-b294-497ca6c416f5 tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:08.174490 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a5a10-703e-476e-b294-497ca6c416f5 tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:08.198613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:08.271358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1264/2512] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:41:08 2022] POST /identity/v3/projects => generated 354 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:08.276387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d75ca8-3e6c-4217-b9eb-b8a7fe4557b4 tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:08.294668 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d75ca8-3e6c-4217-b9eb-b8a7fe4557b4 tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:08.520267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1fb96-a926-4e14-8544-bef5dc23c3db tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fb79c3e45ec44e39cac0b7523f3b45b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebca6edbf09c42debbe919838dab3ba4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:08.521038 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1fb96-a926-4e14-8544-bef5dc23c3db tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:08.521402 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1fb96-a926-4e14-8544-bef5dc23c3db tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:08.523540 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1fb96-a926-4e14-8544-bef5dc23c3db tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:08.524202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1fb96-a926-4e14-8544-bef5dc23c3db tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:41:08.524534 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1fb96-a926-4e14-8544-bef5dc23c3db tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:08.526232 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1fb96-a926-4e14-8544-bef5dc23c3db tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:08.584635 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1249/2513] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:41:08.651495 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a5a10-703e-476e-b294-497ca6c416f5 tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:08.693056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a5a10-703e-476e-b294-497ca6c416f5 tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:08.713438 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba22d9-2b3b-47d7-852c-5d7a1ed079cf tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fb79c3e45ec44e39cac0b7523f3b45b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebca6edbf09c42debbe919838dab3ba4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:08.714768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba22d9-2b3b-47d7-852c-5d7a1ed079cf tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:08.715234 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba22d9-2b3b-47d7-852c-5d7a1ed079cf tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:08.715632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba22d9-2b3b-47d7-852c-5d7a1ed079cf tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] PATH_INFO: `/v3/projects/13e708c37251405f8b8ac1e6834e0f0a` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:08.724767 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba22d9-2b3b-47d7-852c-5d7a1ed079cf tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] RBAC: Authorizing `identity:delete_project(project_id=13e708c37251405f8b8ac1e6834e0f0a)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:08.729701 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba22d9-2b3b-47d7-852c-5d7a1ed079cf tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:08.959692 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-0eba22d9-2b3b-47d7-852c-5d7a1ed079cf tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13e708c37251405f8b8ac1e6834e0f0a', 'request_id': 'req-0eba22d9-2b3b-47d7-852c-5d7a1ed079cf'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:08.990051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-0eba22d9-2b3b-47d7-852c-5d7a1ed079cf tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13e708c37251405f8b8ac1e6834e0f0a', 'request_id': 'req-0eba22d9-2b3b-47d7-852c-5d7a1ed079cf'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:09.017239 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-0eba22d9-2b3b-47d7-852c-5d7a1ed079cf tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13e708c37251405f8b8ac1e6834e0f0a', 'request_id': 'req-0eba22d9-2b3b-47d7-852c-5d7a1ed079cf'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:09.071837 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-0eba22d9-2b3b-47d7-852c-5d7a1ed079cf tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] The token cache is being invalidate because project 13e708c37251405f8b8ac1e6834e0f0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:41:09.075182 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-0eba22d9-2b3b-47d7-852c-5d7a1ed079cf tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:09.076836 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-0eba22d9-2b3b-47d7-852c-5d7a1ed079cf tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:09.089328 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1265/2514] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:41:08 2022] DELETE /identity/v3/projects/13e708c37251405f8b8ac1e6834e0f0a => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:09.719505 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1fb96-a926-4e14-8544-bef5dc23c3db tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:09.722137 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-48c1fb96-a926-4e14-8544-bef5dc23c3db tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:09.886785 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:09.935722 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1fb96-a926-4e14-8544-bef5dc23c3db tempest-VolumeQuotaClassesTest-1789192579 tempest-VolumeQuotaClassesTest-1789192579-project-admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:10.075138 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ce388-e238-432b-94ef-40c1d89b96e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:10.076055 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ce388-e238-432b-94ef-40c1d89b96e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:10.076345 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ce388-e238-432b-94ef-40c1d89b96e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:10.076698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ce388-e238-432b-94ef-40c1d89b96e0 admin admin] PATH_INFO: `/v3/users/523d2917d8214af0b496ae813df81090` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:10.078508 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ce388-e238-432b-94ef-40c1d89b96e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=523d2917d8214af0b496ae813df81090)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:10.080482 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ce388-e238-432b-94ef-40c1d89b96e0 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:10.245639 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e01ce388-e238-432b-94ef-40c1d89b96e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '523d2917d8214af0b496ae813df81090'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:10.273279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e01ce388-e238-432b-94ef-40c1d89b96e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '523d2917d8214af0b496ae813df81090'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:10.276109 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e01ce388-e238-432b-94ef-40c1d89b96e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '523d2917d8214af0b496ae813df81090'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:10.301034 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e01ce388-e238-432b-94ef-40c1d89b96e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '523d2917d8214af0b496ae813df81090'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:10.320565 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e01ce388-e238-432b-94ef-40c1d89b96e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '523d2917d8214af0b496ae813df81090'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:10.346067 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1250/2515] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:41:09 2022] DELETE /identity/v3/users/523d2917d8214af0b496ae813df81090 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:10.350035 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce64ff-3c73-4f65-b96c-9d3185d66805 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:10.350857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce64ff-3c73-4f65-b96c-9d3185d66805 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:10.350857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce64ff-3c73-4f65-b96c-9d3185d66805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:10.422413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-34ce64ff-3c73-4f65-b96c-9d3185d66805 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:10.427682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-34ce64ff-3c73-4f65-b96c-9d3185d66805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:10.823795 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1266/2516] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:10.841192 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ce388-e238-432b-94ef-40c1d89b96e0 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:10.843037 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e01ce388-e238-432b-94ef-40c1d89b96e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:11.092712 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ce388-e238-432b-94ef-40c1d89b96e0 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:11.105975 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4115f-4728-4372-bb1d-d0b6ad94470c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:11.107003 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4115f-4728-4372-bb1d-d0b6ad94470c service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:11.107514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4115f-4728-4372-bb1d-d0b6ad94470c service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:11.108217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4115f-4728-4372-bb1d-d0b6ad94470c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:11.141614 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c4115f-4728-4372-bb1d-d0b6ad94470c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:11.144066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c4115f-4728-4372-bb1d-d0b6ad94470c service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:11.183502 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1251/2517] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:12.446204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce64ff-3c73-4f65-b96c-9d3185d66805 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:12.448738 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-34ce64ff-3c73-4f65-b96c-9d3185d66805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:12.582202 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce64ff-3c73-4f65-b96c-9d3185d66805 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:12.602176 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6666682-3d91-4507-9fb5-e8922785d64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:12.604552 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6666682-3d91-4507-9fb5-e8922785d64f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:12.605056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6666682-3d91-4507-9fb5-e8922785d64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:12.609217 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6666682-3d91-4507-9fb5-e8922785d64f admin admin] PATH_INFO: `/v3/projects/f64c728f2f174894a788ceeea669aa8d` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:12.619333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6666682-3d91-4507-9fb5-e8922785d64f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f64c728f2f174894a788ceeea669aa8d)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:12.630000 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6666682-3d91-4507-9fb5-e8922785d64f admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:12.823929 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-f6666682-3d91-4507-9fb5-e8922785d64f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f64c728f2f174894a788ceeea669aa8d', 'request_id': 'req-f6666682-3d91-4507-9fb5-e8922785d64f'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:12.841797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-f6666682-3d91-4507-9fb5-e8922785d64f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f64c728f2f174894a788ceeea669aa8d', 'request_id': 'req-f6666682-3d91-4507-9fb5-e8922785d64f'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:12.858664 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-f6666682-3d91-4507-9fb5-e8922785d64f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f64c728f2f174894a788ceeea669aa8d', 'request_id': 'req-f6666682-3d91-4507-9fb5-e8922785d64f'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:12.865715 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=84,keystone:INSERT=3,keystone:DELETE=13 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:12.901778 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-f6666682-3d91-4507-9fb5-e8922785d64f admin admin] The token cache is being invalidate because project f64c728f2f174894a788ceeea669aa8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:41:12.902214 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-f6666682-3d91-4507-9fb5-e8922785d64f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:12.911131 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-f6666682-3d91-4507-9fb5-e8922785d64f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:12.916229 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1267/2518] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:41:12 2022] DELETE /identity/v3/projects/f64c728f2f174894a788ceeea669aa8d => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:12.919078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4115f-4728-4372-bb1d-d0b6ad94470c service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:12.939716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4115f-4728-4372-bb1d-d0b6ad94470c service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:13.016046 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:13.187091 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616dbbd-0e6a-4269-bde5-1736c0613de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:13.192258 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616dbbd-0e6a-4269-bde5-1736c0613de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:13.192548 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616dbbd-0e6a-4269-bde5-1736c0613de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:13.192970 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616dbbd-0e6a-4269-bde5-1736c0613de6 admin admin] PATH_INFO: `/v3/users/2fb79c3e45ec44e39cac0b7523f3b45b` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:13.203475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c616dbbd-0e6a-4269-bde5-1736c0613de6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fb79c3e45ec44e39cac0b7523f3b45b)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:13.205481 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c616dbbd-0e6a-4269-bde5-1736c0613de6 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:13.414558 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c616dbbd-0e6a-4269-bde5-1736c0613de6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fb79c3e45ec44e39cac0b7523f3b45b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:13.470160 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c616dbbd-0e6a-4269-bde5-1736c0613de6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fb79c3e45ec44e39cac0b7523f3b45b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:13.472276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c616dbbd-0e6a-4269-bde5-1736c0613de6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fb79c3e45ec44e39cac0b7523f3b45b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:13.513077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c616dbbd-0e6a-4269-bde5-1736c0613de6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fb79c3e45ec44e39cac0b7523f3b45b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:13.544438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c616dbbd-0e6a-4269-bde5-1736c0613de6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fb79c3e45ec44e39cac0b7523f3b45b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:13.565891 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1252/2519] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:41:12 2022] DELETE /identity/v3/users/2fb79c3e45ec44e39cac0b7523f3b45b => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:13.776567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6666682-3d91-4507-9fb5-e8922785d64f admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:13.778986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-f6666682-3d91-4507-9fb5-e8922785d64f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:13.837079 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6666682-3d91-4507-9fb5-e8922785d64f admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:14.031228 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09772ec9-2343-43b5-a1a4-2e76d3fef19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:14.032902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09772ec9-2343-43b5-a1a4-2e76d3fef19c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:14.033525 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09772ec9-2343-43b5-a1a4-2e76d3fef19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:14.034048 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09772ec9-2343-43b5-a1a4-2e76d3fef19c admin admin] PATH_INFO: `/v3/users/209ada6ba12b4bfda10045b4ee9301f1` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:14.035817 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09772ec9-2343-43b5-a1a4-2e76d3fef19c admin admin] RBAC: Authorizing `identity:delete_user(user_id=209ada6ba12b4bfda10045b4ee9301f1)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:14.038360 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09772ec9-2343-43b5-a1a4-2e76d3fef19c admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:14.138371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-09772ec9-2343-43b5-a1a4-2e76d3fef19c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '209ada6ba12b4bfda10045b4ee9301f1'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:14.157325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-09772ec9-2343-43b5-a1a4-2e76d3fef19c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '209ada6ba12b4bfda10045b4ee9301f1'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:14.162185 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-09772ec9-2343-43b5-a1a4-2e76d3fef19c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '209ada6ba12b4bfda10045b4ee9301f1'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:14.217267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-09772ec9-2343-43b5-a1a4-2e76d3fef19c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '209ada6ba12b4bfda10045b4ee9301f1'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:14.253563 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-09772ec9-2343-43b5-a1a4-2e76d3fef19c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '209ada6ba12b4bfda10045b4ee9301f1'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:14.266143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1268/2520] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:41:13 2022] DELETE /identity/v3/users/209ada6ba12b4bfda10045b4ee9301f1 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:14.492527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616dbbd-0e6a-4269-bde5-1736c0613de6 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:14.494672 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c616dbbd-0e6a-4269-bde5-1736c0613de6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:14.532587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616dbbd-0e6a-4269-bde5-1736c0613de6 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:14.691690 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09772ec9-2343-43b5-a1a4-2e76d3fef19c admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:14.691690 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-09772ec9-2343-43b5-a1a4-2e76d3fef19c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:14.750976 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a012b-4ec8-4838-9020-3da8794f1d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:14.753625 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a012b-4ec8-4838-9020-3da8794f1d8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:14.754601 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a012b-4ec8-4838-9020-3da8794f1d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:14.756279 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a012b-4ec8-4838-9020-3da8794f1d8a admin admin] PATH_INFO: `/v3/projects/ebca6edbf09c42debbe919838dab3ba4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:14.758056 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a012b-4ec8-4838-9020-3da8794f1d8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebca6edbf09c42debbe919838dab3ba4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:14.760848 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a012b-4ec8-4838-9020-3da8794f1d8a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:14.845319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:14.867920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09772ec9-2343-43b5-a1a4-2e76d3fef19c admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:14.870398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3f3a012b-4ec8-4838-9020-3da8794f1d8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebca6edbf09c42debbe919838dab3ba4', 'request_id': 'req-3f3a012b-4ec8-4838-9020-3da8794f1d8a'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:14.887109 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3f3a012b-4ec8-4838-9020-3da8794f1d8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebca6edbf09c42debbe919838dab3ba4', 'request_id': 'req-3f3a012b-4ec8-4838-9020-3da8794f1d8a'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:14.907466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:14.908985 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3f3a012b-4ec8-4838-9020-3da8794f1d8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebca6edbf09c42debbe919838dab3ba4', 'request_id': 'req-3f3a012b-4ec8-4838-9020-3da8794f1d8a'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:14.954930 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3f3a012b-4ec8-4838-9020-3da8794f1d8a admin admin] The token cache is being invalidate because project ebca6edbf09c42debbe919838dab3ba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:41:14.956923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3f3a012b-4ec8-4838-9020-3da8794f1d8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:14.960703 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-3f3a012b-4ec8-4838-9020-3da8794f1d8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:14.972187 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1253/2521] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:41:14 2022] DELETE /identity/v3/projects/ebca6edbf09c42debbe919838dab3ba4 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:14.977508 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3c1e1-452f-4ee4-9f09-f54dc416c37c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:14.978986 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3c1e1-452f-4ee4-9f09-f54dc416c37c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:14.981393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3c1e1-452f-4ee4-9f09-f54dc416c37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:14.999639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b1ad3-2e7c-4a13-8040-1d59fa295885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:15.000557 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b1ad3-2e7c-4a13-8040-1d59fa295885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:15.000959 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b1ad3-2e7c-4a13-8040-1d59fa295885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:15.001346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b1ad3-2e7c-4a13-8040-1d59fa295885 admin admin] PATH_INFO: `/v3/projects/d63c592c0afb4dfc9dd5b36be791decd` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:15.002971 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b1ad3-2e7c-4a13-8040-1d59fa295885 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d63c592c0afb4dfc9dd5b36be791decd)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:15.005454 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b1ad3-2e7c-4a13-8040-1d59fa295885 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:15.069522 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-81f3c1e1-452f-4ee4-9f09-f54dc416c37c None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:15.127018 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-81f3c1e1-452f-4ee4-9f09-f54dc416c37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:15.175876 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1254/2522] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:15.183566 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178fd69e-568a-4b5e-b384-83ae0055e237 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:15.186922 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178fd69e-568a-4b5e-b384-83ae0055e237 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:15.187460 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178fd69e-568a-4b5e-b384-83ae0055e237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:15.255524 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-178fd69e-568a-4b5e-b384-83ae0055e237 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:15.260020 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-178fd69e-568a-4b5e-b384-83ae0055e237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:15.271432 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1255/2523] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:15.275464 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178fd69e-568a-4b5e-b384-83ae0055e237 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:15.310870 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178fd69e-568a-4b5e-b384-83ae0055e237 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:15.334051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-042b1ad3-2e7c-4a13-8040-1d59fa295885 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd63c592c0afb4dfc9dd5b36be791decd', 'request_id': 'req-042b1ad3-2e7c-4a13-8040-1d59fa295885'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:15.345957 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-042b1ad3-2e7c-4a13-8040-1d59fa295885 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd63c592c0afb4dfc9dd5b36be791decd', 'request_id': 'req-042b1ad3-2e7c-4a13-8040-1d59fa295885'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:15.355473 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-042b1ad3-2e7c-4a13-8040-1d59fa295885 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd63c592c0afb4dfc9dd5b36be791decd', 'request_id': 'req-042b1ad3-2e7c-4a13-8040-1d59fa295885'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:15.379500 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-042b1ad3-2e7c-4a13-8040-1d59fa295885 admin admin] The token cache is being invalidate because project d63c592c0afb4dfc9dd5b36be791decd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:41:15.379825 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-042b1ad3-2e7c-4a13-8040-1d59fa295885 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:15.380531 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-042b1ad3-2e7c-4a13-8040-1d59fa295885 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:15.382532 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1269/2524] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:41:14 2022] DELETE /identity/v3/projects/d63c592c0afb4dfc9dd5b36be791decd => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:15.388013 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4fcda-d87f-4afc-a162-b61c897cc207 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:15.388384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4fcda-d87f-4afc-a162-b61c897cc207 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:15.388588 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4fcda-d87f-4afc-a162-b61c897cc207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:15.442708 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-40f4fcda-d87f-4afc-a162-b61c897cc207 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:15.446428 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-40f4fcda-d87f-4afc-a162-b61c897cc207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:15.449874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674c394-4da8-4515-8ad3-0c91640dfc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:15.450805 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674c394-4da8-4515-8ad3-0c91640dfc56 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:15.451850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674c394-4da8-4515-8ad3-0c91640dfc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:15.453056 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674c394-4da8-4515-8ad3-0c91640dfc56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:15.456250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674c394-4da8-4515-8ad3-0c91640dfc56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:41:15.457253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674c394-4da8-4515-8ad3-0c91640dfc56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:15.460509 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674c394-4da8-4515-8ad3-0c91640dfc56 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:15.495846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1256/2525] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:41:15.497215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674c394-4da8-4515-8ad3-0c91640dfc56 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:15.499662 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6674c394-4da8-4515-8ad3-0c91640dfc56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:15.517812 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1270/2526] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:15.521079 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cea1b-42d2-445f-8927-50003589d6bb None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:15.521889 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cea1b-42d2-445f-8927-50003589d6bb None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:15.522188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cea1b-42d2-445f-8927-50003589d6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:15.532524 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674c394-4da8-4515-8ad3-0c91640dfc56 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:15.547183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6d20f-3afa-4cd2-abe6-24713e72b995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:15.548120 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6d20f-3afa-4cd2-abe6-24713e72b995 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:15.548624 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6d20f-3afa-4cd2-abe6-24713e72b995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:15.548963 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6d20f-3afa-4cd2-abe6-24713e72b995 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:15.552917 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6d20f-3afa-4cd2-abe6-24713e72b995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:15.557488 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-842cea1b-42d2-445f-8927-50003589d6bb None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:15.558681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6d20f-3afa-4cd2-abe6-24713e72b995 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:15.559156 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-842cea1b-42d2-445f-8927-50003589d6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:15.588546 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1271/2527] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:15.589980 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cea1b-42d2-445f-8927-50003589d6bb None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:15.619518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1257/2528] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:41:15 2022] POST /identity/v3/projects => generated 365 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:15.619771 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cea1b-42d2-445f-8927-50003589d6bb None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:15.623912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6d20f-3afa-4cd2-abe6-24713e72b995 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:15.730713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6d20f-3afa-4cd2-abe6-24713e72b995 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:15.738203 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56ea99-e5da-4b42-8222-98bc73032c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:15.738982 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56ea99-e5da-4b42-8222-98bc73032c46 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:15.739322 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56ea99-e5da-4b42-8222-98bc73032c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:15.739593 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56ea99-e5da-4b42-8222-98bc73032c46 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:15.740387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b56ea99-e5da-4b42-8222-98bc73032c46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:15.754572 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b56ea99-e5da-4b42-8222-98bc73032c46 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:15.778084 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cdf3d-7390-4b30-b9a7-accf43d70f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:15.778616 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cdf3d-7390-4b30-b9a7-accf43d70f6d admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:15.779366 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cdf3d-7390-4b30-b9a7-accf43d70f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:15.779716 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cdf3d-7390-4b30-b9a7-accf43d70f6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:15.781609 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922cdf3d-7390-4b30-b9a7-accf43d70f6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:41:15.781962 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922cdf3d-7390-4b30-b9a7-accf43d70f6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:15.783766 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922cdf3d-7390-4b30-b9a7-accf43d70f6d admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:15.807321 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1272/2529] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:41:15.809202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1258/2530] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:41:15 2022] POST /identity/v3/users => generated 283 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:15.813117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56ea99-e5da-4b42-8222-98bc73032c46 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:15.814809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cdf3d-7390-4b30-b9a7-accf43d70f6d admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:15.829411 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cdf3d-7390-4b30-b9a7-accf43d70f6d admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:15.835829 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744aea2-12ae-4242-afc5-6cf4b83ec5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:15.837318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744aea2-12ae-4242-afc5-6cf4b83ec5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:15.837822 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744aea2-12ae-4242-afc5-6cf4b83ec5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:15.838323 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744aea2-12ae-4242-afc5-6cf4b83ec5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:15.839349 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f744aea2-12ae-4242-afc5-6cf4b83ec5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:15.841566 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f744aea2-12ae-4242-afc5-6cf4b83ec5a2 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:15.847163 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56ea99-e5da-4b42-8222-98bc73032c46 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:15.859941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1273/2531] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:15.863985 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecfe47-cb02-48f2-b3d2-d7b428e35271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:15.864712 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecfe47-cb02-48f2-b3d2-d7b428e35271 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:15.865003 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecfe47-cb02-48f2-b3d2-d7b428e35271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:15.865314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecfe47-cb02-48f2-b3d2-d7b428e35271 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:15.865675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744aea2-12ae-4242-afc5-6cf4b83ec5a2 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:15.866030 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ecfe47-cb02-48f2-b3d2-d7b428e35271 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:15.874833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ecfe47-cb02-48f2-b3d2-d7b428e35271 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:15.885365 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744aea2-12ae-4242-afc5-6cf4b83ec5a2 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:15.898804 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75127f1f-e372-4751-89a4-8972f1c93bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:15.903363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75127f1f-e372-4751-89a4-8972f1c93bec admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:15.904822 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75127f1f-e372-4751-89a4-8972f1c93bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:15.905711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75127f1f-e372-4751-89a4-8972f1c93bec admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:15.907652 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75127f1f-e372-4751-89a4-8972f1c93bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:15.911285 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75127f1f-e372-4751-89a4-8972f1c93bec admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:15.921217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:15.933396 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1274/2532] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:15.937644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75127f1f-e372-4751-89a4-8972f1c93bec admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:15.957325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75127f1f-e372-4751-89a4-8972f1c93bec admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:15.963394 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1259/2533] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:41:15 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:15.968499 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecfe47-cb02-48f2-b3d2-d7b428e35271 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:15.972839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d02fb-479a-4f12-ab15-38883c0013ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:15.973793 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d02fb-479a-4f12-ab15-38883c0013ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:15.974346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d02fb-479a-4f12-ab15-38883c0013ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:15.974635 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d02fb-479a-4f12-ab15-38883c0013ed admin admin] PATH_INFO: `/v3/projects/0e4b38712dee4ee09e9a7492d2b9a0a2/users/4f6aa96da1024ad5ac2f9e947b26e491/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:15.985344 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:16.040186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d02fb-479a-4f12-ab15-38883c0013ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e4b38712dee4ee09e9a7492d2b9a0a2, user_id=4f6aa96da1024ad5ac2f9e947b26e491, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:16.044238 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d02fb-479a-4f12-ab15-38883c0013ed admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:16.063886 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecfe47-cb02-48f2-b3d2-d7b428e35271 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:16.070143 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c30445b-db6c-41ee-981e-1284eb68371b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:16.073335 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30445b-db6c-41ee-981e-1284eb68371b admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:16.078717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1275/2534] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:15 2022] PUT /identity/v3/projects/0e4b38712dee4ee09e9a7492d2b9a0a2/users/4f6aa96da1024ad5ac2f9e947b26e491/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:16.091399 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb54d6-859d-4399-9f9b-dbb5d3d45bde None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:16.091694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb54d6-859d-4399-9f9b-dbb5d3d45bde None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:16.091984 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb54d6-859d-4399-9f9b-dbb5d3d45bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:16.093288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30445b-db6c-41ee-981e-1284eb68371b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:16.093718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30445b-db6c-41ee-981e-1284eb68371b admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:16.094558 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c30445b-db6c-41ee-981e-1284eb68371b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:16.097525 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c30445b-db6c-41ee-981e-1284eb68371b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:16.145557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1260/2535] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:41:15 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:41:16.150455 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c30445b-db6c-41ee-981e-1284eb68371b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:16.164118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-19fb54d6-859d-4399-9f9b-dbb5d3d45bde None None] MFA Rules not processed for user `4f6aa96da1024ad5ac2f9e947b26e491`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:16.166704 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-19fb54d6-859d-4399-9f9b-dbb5d3d45bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:16.170573 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c30445b-db6c-41ee-981e-1284eb68371b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:16.301753 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c871aab-a3f4-476f-8d59-743ccf48ae15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:16.302833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c871aab-a3f4-476f-8d59-743ccf48ae15 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:16.303119 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c871aab-a3f4-476f-8d59-743ccf48ae15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:16.303406 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c871aab-a3f4-476f-8d59-743ccf48ae15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:16.304163 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c871aab-a3f4-476f-8d59-743ccf48ae15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:16.305829 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c871aab-a3f4-476f-8d59-743ccf48ae15 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:16.326591 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1261/2536] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:16 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:16.330193 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c871aab-a3f4-476f-8d59-743ccf48ae15 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:16.344201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1276/2537] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:16 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:16.348414 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb54d6-859d-4399-9f9b-dbb5d3d45bde None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:16.364828 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c871aab-a3f4-476f-8d59-743ccf48ae15 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:16.376562 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb54d6-859d-4399-9f9b-dbb5d3d45bde None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:16.380887 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e7079-51bf-4bc9-a1a6-2621a2525dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:16.383096 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e7079-51bf-4bc9-a1a6-2621a2525dcc admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:16.384991 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e7079-51bf-4bc9-a1a6-2621a2525dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:16.386810 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e7079-51bf-4bc9-a1a6-2621a2525dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:16.389453 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1e7079-51bf-4bc9-a1a6-2621a2525dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:16.393973 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1e7079-51bf-4bc9-a1a6-2621a2525dcc admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:16.398939 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a4a6b-b731-48a1-b70e-ec5b636569df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:16.400039 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a4a6b-b731-48a1-b70e-ec5b636569df admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:16.402639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a4a6b-b731-48a1-b70e-ec5b636569df admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:16.403790 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a4a6b-b731-48a1-b70e-ec5b636569df admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:16.408736 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a4a6b-b731-48a1-b70e-ec5b636569df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:16.414988 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a4a6b-b731-48a1-b70e-ec5b636569df admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:16.433210 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1262/2538] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:16 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:16.441828 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e7079-51bf-4bc9-a1a6-2621a2525dcc admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:16.468688 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e7079-51bf-4bc9-a1a6-2621a2525dcc admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:16.477242 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b384082-db39-449c-9bdd-2a47d9efad92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:16.478578 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b384082-db39-449c-9bdd-2a47d9efad92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:16.479270 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b384082-db39-449c-9bdd-2a47d9efad92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:16.481406 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b384082-db39-449c-9bdd-2a47d9efad92 admin admin] PATH_INFO: `/v3/projects/c7add0887fb448cb8f95761b746c5f00/users/d0422683c5af464b8361d211a19f50ba/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:16.492424 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1277/2539] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:41:16 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:41:16.497591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a4a6b-b731-48a1-b70e-ec5b636569df admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:16.524949 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b384082-db39-449c-9bdd-2a47d9efad92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7add0887fb448cb8f95761b746c5f00, user_id=d0422683c5af464b8361d211a19f50ba, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:16.535839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b384082-db39-449c-9bdd-2a47d9efad92 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:16.565991 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1263/2540] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:16 2022] PUT /identity/v3/projects/c7add0887fb448cb8f95761b746c5f00/users/d0422683c5af464b8361d211a19f50ba/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:16.582815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcc848-f81c-4dc2-a6b6-e6489ed14f9b None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:16.585446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcc848-f81c-4dc2-a6b6-e6489ed14f9b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:16.585924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcc848-f81c-4dc2-a6b6-e6489ed14f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:16.636816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-45dcc848-f81c-4dc2-a6b6-e6489ed14f9b None None] MFA Rules not processed for user `d0422683c5af464b8361d211a19f50ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:16.638481 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-45dcc848-f81c-4dc2-a6b6-e6489ed14f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:16.638637 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a4a6b-b731-48a1-b70e-ec5b636569df admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:16.706859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:16.786182 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1264/2541] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:16.810994 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dcc848-f81c-4dc2-a6b6-e6489ed14f9b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:16.812601 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-45dcc848-f81c-4dc2-a6b6-e6489ed14f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:16.820535 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81f5f9-c663-40ca-a6f1-80696897486f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:16.821445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81f5f9-c663-40ca-a6f1-80696897486f admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:16.821740 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81f5f9-c663-40ca-a6f1-80696897486f admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:16.822028 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81f5f9-c663-40ca-a6f1-80696897486f admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:16.824120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81f5f9-c663-40ca-a6f1-80696897486f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:16.827109 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81f5f9-c663-40ca-a6f1-80696897486f admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:16.844977 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:16.924141 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1278/2542] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:41:16 2022] POST /identity/v3/users => generated 289 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:16.930383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81f5f9-c663-40ca-a6f1-80696897486f admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:16.969478 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81f5f9-c663-40ca-a6f1-80696897486f admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:16.993097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4c4ec-2029-4dc2-9da2-b43ebc6e136c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:16.993853 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4c4ec-2029-4dc2-9da2-b43ebc6e136c admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:16.994182 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4c4ec-2029-4dc2-9da2-b43ebc6e136c admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:16.994477 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4c4ec-2029-4dc2-9da2-b43ebc6e136c admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:16.995298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf4c4ec-2029-4dc2-9da2-b43ebc6e136c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:16.996862 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf4c4ec-2029-4dc2-9da2-b43ebc6e136c admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:17.025630 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1279/2543] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:16 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:17.028442 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4c4ec-2029-4dc2-9da2-b43ebc6e136c admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:17.042407 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4c4ec-2029-4dc2-9da2-b43ebc6e136c admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:17.049729 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d5dc2-1d8c-433a-9b43-2523e7d1a31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:17.050949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d5dc2-1d8c-433a-9b43-2523e7d1a31b admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:17.051485 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d5dc2-1d8c-433a-9b43-2523e7d1a31b admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:17.051900 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d5dc2-1d8c-433a-9b43-2523e7d1a31b admin admin] PATH_INFO: `/v3/projects/a4c1176d2276459e8aba881b8be92eb2/users/cc0a6edcb2134665baf7b9a20a2eed50/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:17.071413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d5dc2-1d8c-433a-9b43-2523e7d1a31b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4c1176d2276459e8aba881b8be92eb2, user_id=cc0a6edcb2134665baf7b9a20a2eed50, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:17.074579 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d5dc2-1d8c-433a-9b43-2523e7d1a31b admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:17.075850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dcc848-f81c-4dc2-a6b6-e6489ed14f9b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:17.101186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1280/2544] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:17 2022] PUT /identity/v3/projects/a4c1176d2276459e8aba881b8be92eb2/users/cc0a6edcb2134665baf7b9a20a2eed50/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:17.101812 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc713f41-f16c-4a2a-bd89-1bbaed34ebf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:17.102515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc713f41-f16c-4a2a-bd89-1bbaed34ebf2 service nova] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:17.102811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc713f41-f16c-4a2a-bd89-1bbaed34ebf2 service nova] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:17.103095 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc713f41-f16c-4a2a-bd89-1bbaed34ebf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:17.104756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d5dc2-1d8c-433a-9b43-2523e7d1a31b admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:17.116428 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc713f41-f16c-4a2a-bd89-1bbaed34ebf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:17.118120 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc713f41-f16c-4a2a-bd89-1bbaed34ebf2 service nova] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:17.132917 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1265/2545] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:41:17.263567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d5dc2-1d8c-433a-9b43-2523e7d1a31b admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:17.281391 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233db4d7-9f7c-4371-81a7-934fb07e47f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:17.282100 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233db4d7-9f7c-4371-81a7-934fb07e47f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:17.282398 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233db4d7-9f7c-4371-81a7-934fb07e47f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:17.282687 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233db4d7-9f7c-4371-81a7-934fb07e47f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:17.283665 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233db4d7-9f7c-4371-81a7-934fb07e47f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:17.285398 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233db4d7-9f7c-4371-81a7-934fb07e47f4 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:17.314301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1281/2546] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:17.318842 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc713f41-f16c-4a2a-bd89-1bbaed34ebf2 service nova] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:17.343691 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc713f41-f16c-4a2a-bd89-1bbaed34ebf2 service nova] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:17.355768 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd340f-886b-432b-85d6-3ff56abc69f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:17.357241 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd340f-886b-432b-85d6-3ff56abc69f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:17.357982 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd340f-886b-432b-85d6-3ff56abc69f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:17.359489 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd340f-886b-432b-85d6-3ff56abc69f4 admin admin] PATH_INFO: `/v3/projects/a4c1176d2276459e8aba881b8be92eb2/users/cc0a6edcb2134665baf7b9a20a2eed50/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:17.363672 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd340f-886b-432b-85d6-3ff56abc69f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4c1176d2276459e8aba881b8be92eb2, user_id=cc0a6edcb2134665baf7b9a20a2eed50, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:17.367437 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd340f-886b-432b-85d6-3ff56abc69f4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:17.387483 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1266/2547] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:17 2022] PUT /identity/v3/projects/a4c1176d2276459e8aba881b8be92eb2/users/cc0a6edcb2134665baf7b9a20a2eed50/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:17.402093 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199519d2-816e-4645-bade-a94c3ff95ff9 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:17.402399 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199519d2-816e-4645-bade-a94c3ff95ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:17.402744 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199519d2-816e-4645-bade-a94c3ff95ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:17.464160 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-199519d2-816e-4645-bade-a94c3ff95ff9 None None] MFA Rules not processed for user `cc0a6edcb2134665baf7b9a20a2eed50`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:17.465631 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-199519d2-816e-4645-bade-a94c3ff95ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:17.649947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1282/2548] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:17 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:17.674042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd340f-886b-432b-85d6-3ff56abc69f4 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:17.676150 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-52dd340f-886b-432b-85d6-3ff56abc69f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:17.877767 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd340f-886b-432b-85d6-3ff56abc69f4 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:17.882402 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b707419-6b25-4ce3-bcaf-96aeda62f9f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:17.883507 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b707419-6b25-4ce3-bcaf-96aeda62f9f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:17.883954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b707419-6b25-4ce3-bcaf-96aeda62f9f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:17.884496 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b707419-6b25-4ce3-bcaf-96aeda62f9f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:17.898893 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b707419-6b25-4ce3-bcaf-96aeda62f9f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:17.900685 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b707419-6b25-4ce3-bcaf-96aeda62f9f6 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:17.919155 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1267/2549] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:17 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:18.047903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199519d2-816e-4645-bade-a94c3ff95ff9 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:18.071854 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199519d2-816e-4645-bade-a94c3ff95ff9 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:18.083819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849b74f-8fc4-4e98-8e88-bdb293332a8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:18.086800 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849b74f-8fc4-4e98-8e88-bdb293332a8c service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:18.087019 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849b74f-8fc4-4e98-8e88-bdb293332a8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:18.087369 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849b74f-8fc4-4e98-8e88-bdb293332a8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:18.176986 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:18.277253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849b74f-8fc4-4e98-8e88-bdb293332a8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:18.279359 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849b74f-8fc4-4e98-8e88-bdb293332a8c service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:18.301725 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1283/2550] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:18 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:21.652698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c08df-f4bd-4751-822b-59360356f2ef None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:21.653487 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c08df-f4bd-4751-822b-59360356f2ef None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:21.653487 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c08df-f4bd-4751-822b-59360356f2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:21.676424 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-888c08df-f4bd-4751-822b-59360356f2ef None None] MFA Rules not processed for user `af7be60f65da43f982c1ec2e91397051`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:21.677846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-888c08df-f4bd-4751-822b-59360356f2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:21.733504 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:21.748646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1268/2551] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:23.535668 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849b74f-8fc4-4e98-8e88-bdb293332a8c service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:23.646661 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849b74f-8fc4-4e98-8e88-bdb293332a8c service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:23.649345 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9355c-7f99-4a1d-ba52-0a74eccb651f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:23.650338 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9355c-7f99-4a1d-ba52-0a74eccb651f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:23.650861 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9355c-7f99-4a1d-ba52-0a74eccb651f admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:23.651169 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9355c-7f99-4a1d-ba52-0a74eccb651f admin admin] PATH_INFO: `/v3/users/4f6aa96da1024ad5ac2f9e947b26e491` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:23.652691 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c9355c-7f99-4a1d-ba52-0a74eccb651f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f6aa96da1024ad5ac2f9e947b26e491)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:23.655868 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c9355c-7f99-4a1d-ba52-0a74eccb651f admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:23.741979 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-11c9355c-7f99-4a1d-ba52-0a74eccb651f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f6aa96da1024ad5ac2f9e947b26e491'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:23.756412 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-11c9355c-7f99-4a1d-ba52-0a74eccb651f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f6aa96da1024ad5ac2f9e947b26e491'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:23.757673 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-11c9355c-7f99-4a1d-ba52-0a74eccb651f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f6aa96da1024ad5ac2f9e947b26e491'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:23.771216 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-11c9355c-7f99-4a1d-ba52-0a74eccb651f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f6aa96da1024ad5ac2f9e947b26e491'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:23.792264 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-11c9355c-7f99-4a1d-ba52-0a74eccb651f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f6aa96da1024ad5ac2f9e947b26e491'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:23.803182 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1284/2552] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:41:23 2022] DELETE /identity/v3/users/4f6aa96da1024ad5ac2f9e947b26e491 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:23.809947 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648c5b1-02ef-4860-b4e8-0da51c371248 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:23.814033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648c5b1-02ef-4860-b4e8-0da51c371248 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:23.815743 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648c5b1-02ef-4860-b4e8-0da51c371248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:23.851758 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-2648c5b1-02ef-4860-b4e8-0da51c371248 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:23.853348 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2648c5b1-02ef-4860-b4e8-0da51c371248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:23.985212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1269/2553] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:41:24.002853 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9355c-7f99-4a1d-ba52-0a74eccb651f admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:24.005457 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-11c9355c-7f99-4a1d-ba52-0a74eccb651f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:24.059089 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5f4c3-e7be-4c88-96c4-ad7c88cf8a2f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:24.059555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5f4c3-e7be-4c88-96c4-ad7c88cf8a2f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:24.059855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5f4c3-e7be-4c88-96c4-ad7c88cf8a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:24.105971 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9355c-7f99-4a1d-ba52-0a74eccb651f admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:24.111694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1c201-a379-4f16-862f-918e62087398 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:24.112514 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1c201-a379-4f16-862f-918e62087398 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:24.112848 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1c201-a379-4f16-862f-918e62087398 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:24.113268 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1c201-a379-4f16-862f-918e62087398 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:24.117361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0bc5f4c3-e7be-4c88-96c4-ad7c88cf8a2f None None] MFA Rules not processed for user `cfc2f31f654e486d8505d80aad17d17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:24.118658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0bc5f4c3-e7be-4c88-96c4-ad7c88cf8a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:24.124529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:24.131252 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c1c201-a379-4f16-862f-918e62087398 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:24.134475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c1c201-a379-4f16-862f-918e62087398 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:24.159044 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1285/2554] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:24.226959 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1270/2555] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 17:41:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:41:24.247025 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1c201-a379-4f16-862f-918e62087398 service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:24.248658 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-e4c1c201-a379-4f16-862f-918e62087398 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:24.353281 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1c201-a379-4f16-862f-918e62087398 service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:24.360953 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e8e26-0489-49ec-b857-5541eed28cb5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a37d1201d54afdad9b228ef4f2d787', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:24.361801 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e8e26-0489-49ec-b857-5541eed28cb5 service swift] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:24.362102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e8e26-0489-49ec-b857-5541eed28cb5 service swift] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:24.362549 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e8e26-0489-49ec-b857-5541eed28cb5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:24.372280 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e8e26-0489-49ec-b857-5541eed28cb5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:24.374166 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e8e26-0489-49ec-b857-5541eed28cb5 service swift] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:24.385116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1286/2556] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 17:41:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:41:25.342143 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5f4c3-e7be-4c88-96c4-ad7c88cf8a2f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:25.344622 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0bc5f4c3-e7be-4c88-96c4-ad7c88cf8a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:25.379928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5f4c3-e7be-4c88-96c4-ad7c88cf8a2f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:25.396573 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7fc6a-cf15-4df2-8ef5-4fa4592ba40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:25.399631 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7fc6a-cf15-4df2-8ef5-4fa4592ba40f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:25.400025 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7fc6a-cf15-4df2-8ef5-4fa4592ba40f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:25.402407 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7fc6a-cf15-4df2-8ef5-4fa4592ba40f admin admin] PATH_INFO: `/v3/projects/0e4b38712dee4ee09e9a7492d2b9a0a2` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:25.407817 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c7fc6a-cf15-4df2-8ef5-4fa4592ba40f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e4b38712dee4ee09e9a7492d2b9a0a2)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:25.415628 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c7fc6a-cf15-4df2-8ef5-4fa4592ba40f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:25.475486 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e8e26-0489-49ec-b857-5541eed28cb5 service swift] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:25.475486 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-dd4e8e26-0489-49ec-b857-5541eed28cb5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:25.547920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e8e26-0489-49ec-b857-5541eed28cb5 service swift] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:25.554092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-77c7fc6a-cf15-4df2-8ef5-4fa4592ba40f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e4b38712dee4ee09e9a7492d2b9a0a2', 'request_id': 'req-77c7fc6a-cf15-4df2-8ef5-4fa4592ba40f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:25.571836 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-77c7fc6a-cf15-4df2-8ef5-4fa4592ba40f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e4b38712dee4ee09e9a7492d2b9a0a2', 'request_id': 'req-77c7fc6a-cf15-4df2-8ef5-4fa4592ba40f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:25.588199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-77c7fc6a-cf15-4df2-8ef5-4fa4592ba40f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e4b38712dee4ee09e9a7492d2b9a0a2', 'request_id': 'req-77c7fc6a-cf15-4df2-8ef5-4fa4592ba40f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:25.617281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-77c7fc6a-cf15-4df2-8ef5-4fa4592ba40f admin admin] The token cache is being invalidate because project 0e4b38712dee4ee09e9a7492d2b9a0a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:41:25.617855 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-77c7fc6a-cf15-4df2-8ef5-4fa4592ba40f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:25.618832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-77c7fc6a-cf15-4df2-8ef5-4fa4592ba40f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:25.623029 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1271/2557] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:41:25 2022] DELETE /identity/v3/projects/0e4b38712dee4ee09e9a7492d2b9a0a2 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:25.629538 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7fc6a-cf15-4df2-8ef5-4fa4592ba40f admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:25.632182 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-77c7fc6a-cf15-4df2-8ef5-4fa4592ba40f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:25.706935 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:25.799129 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c0ae0-d4b5-4d83-add6-f72e055c08fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:25.806300 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c0ae0-d4b5-4d83-add6-f72e055c08fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:25.806746 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c0ae0-d4b5-4d83-add6-f72e055c08fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:25.807774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c0ae0-d4b5-4d83-add6-f72e055c08fd admin admin] PATH_INFO: `/v3/users/d0422683c5af464b8361d211a19f50ba` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:25.812174 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c0ae0-d4b5-4d83-add6-f72e055c08fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0422683c5af464b8361d211a19f50ba)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:25.818225 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c0ae0-d4b5-4d83-add6-f72e055c08fd admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:25.863811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7fc6a-cf15-4df2-8ef5-4fa4592ba40f admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:25.899590 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b3c96-1099-499e-8d02-b17e87311746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:25.901044 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b3c96-1099-499e-8d02-b17e87311746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:25.915672 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b3c96-1099-499e-8d02-b17e87311746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:25.916004 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b3c96-1099-499e-8d02-b17e87311746 admin admin] PATH_INFO: `/v3/users/cc0a6edcb2134665baf7b9a20a2eed50` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:25.922363 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b3c96-1099-499e-8d02-b17e87311746 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc0a6edcb2134665baf7b9a20a2eed50)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:25.925694 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b3c96-1099-499e-8d02-b17e87311746 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:25.969890 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-cf4c0ae0-d4b5-4d83-add6-f72e055c08fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0422683c5af464b8361d211a19f50ba'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:26.002440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-cf4c0ae0-d4b5-4d83-add6-f72e055c08fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0422683c5af464b8361d211a19f50ba'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:26.003822 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-cf4c0ae0-d4b5-4d83-add6-f72e055c08fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0422683c5af464b8361d211a19f50ba'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:26.027388 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-cf4c0ae0-d4b5-4d83-add6-f72e055c08fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0422683c5af464b8361d211a19f50ba'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:26.066203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-cf4c0ae0-d4b5-4d83-add6-f72e055c08fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0422683c5af464b8361d211a19f50ba'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:26.081391 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1287/2558] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:41:25 2022] DELETE /identity/v3/users/d0422683c5af464b8361d211a19f50ba => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:26.083429 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317b951-e303-4086-baad-fc09af822f15 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:26.083858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317b951-e303-4086-baad-fc09af822f15 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:26.084255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317b951-e303-4086-baad-fc09af822f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:26.087734 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-266b3c96-1099-499e-8d02-b17e87311746 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc0a6edcb2134665baf7b9a20a2eed50'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:26.106113 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:26.124522 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-266b3c96-1099-499e-8d02-b17e87311746 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc0a6edcb2134665baf7b9a20a2eed50'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:26.128484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-266b3c96-1099-499e-8d02-b17e87311746 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc0a6edcb2134665baf7b9a20a2eed50'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:26.131619 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-e317b951-e303-4086-baad-fc09af822f15 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:26.134797 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-e317b951-e303-4086-baad-fc09af822f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:26.150565 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-266b3c96-1099-499e-8d02-b17e87311746 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc0a6edcb2134665baf7b9a20a2eed50'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:26.163296 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-266b3c96-1099-499e-8d02-b17e87311746 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc0a6edcb2134665baf7b9a20a2eed50'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:26.190533 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1272/2559] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:41:25 2022] DELETE /identity/v3/users/cc0a6edcb2134665baf7b9a20a2eed50 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:26.312116 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1288/2560] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:26.340165 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b3c96-1099-499e-8d02-b17e87311746 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:26.343079 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-266b3c96-1099-499e-8d02-b17e87311746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:26.608463 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b3c96-1099-499e-8d02-b17e87311746 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:26.624368 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd7bce-2f40-423e-b8d1-48c8aa3eb001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:26.625426 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd7bce-2f40-423e-b8d1-48c8aa3eb001 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:26.627499 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd7bce-2f40-423e-b8d1-48c8aa3eb001 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:26.628295 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd7bce-2f40-423e-b8d1-48c8aa3eb001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:26.818496 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:26.838005 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd7bce-2f40-423e-b8d1-48c8aa3eb001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:26.840869 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd7bce-2f40-423e-b8d1-48c8aa3eb001 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:26.877179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1273/2561] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:26.925930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317b951-e303-4086-baad-fc09af822f15 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:26.927545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-e317b951-e303-4086-baad-fc09af822f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:26.964059 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317b951-e303-4086-baad-fc09af822f15 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:26.973223 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6b8f5-4e19-4ce2-a5b6-9ca73b1944e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:26.974826 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6b8f5-4e19-4ce2-a5b6-9ca73b1944e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:26.975177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6b8f5-4e19-4ce2-a5b6-9ca73b1944e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:26.975570 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6b8f5-4e19-4ce2-a5b6-9ca73b1944e0 admin admin] PATH_INFO: `/v3/projects/a4c1176d2276459e8aba881b8be92eb2` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:26.977988 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6b8f5-4e19-4ce2-a5b6-9ca73b1944e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4c1176d2276459e8aba881b8be92eb2)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:26.985772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6b8f5-4e19-4ce2-a5b6-9ca73b1944e0 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:27.126061 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a2e6b8f5-4e19-4ce2-a5b6-9ca73b1944e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4c1176d2276459e8aba881b8be92eb2', 'request_id': 'req-a2e6b8f5-4e19-4ce2-a5b6-9ca73b1944e0'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:27.140208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a2e6b8f5-4e19-4ce2-a5b6-9ca73b1944e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4c1176d2276459e8aba881b8be92eb2', 'request_id': 'req-a2e6b8f5-4e19-4ce2-a5b6-9ca73b1944e0'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:27.153642 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a2e6b8f5-4e19-4ce2-a5b6-9ca73b1944e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4c1176d2276459e8aba881b8be92eb2', 'request_id': 'req-a2e6b8f5-4e19-4ce2-a5b6-9ca73b1944e0'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:27.184101 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a2e6b8f5-4e19-4ce2-a5b6-9ca73b1944e0 admin admin] The token cache is being invalidate because project a4c1176d2276459e8aba881b8be92eb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:41:27.184499 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a2e6b8f5-4e19-4ce2-a5b6-9ca73b1944e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:27.185527 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a2e6b8f5-4e19-4ce2-a5b6-9ca73b1944e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:27.188664 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1289/2562] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:41:26 2022] DELETE /identity/v3/projects/a4c1176d2276459e8aba881b8be92eb2 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:27.194814 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aeb744-84b6-4d22-846e-e19077b58d2f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:27.195325 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aeb744-84b6-4d22-846e-e19077b58d2f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:27.195742 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aeb744-84b6-4d22-846e-e19077b58d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:27.256940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-c1aeb744-84b6-4d22-846e-e19077b58d2f None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:27.258964 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c1aeb744-84b6-4d22-846e-e19077b58d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:27.475264 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1274/2563] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:27.484291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782a53c-a70a-4f75-8ab0-a9c3d8756885 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:27.485044 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782a53c-a70a-4f75-8ab0-a9c3d8756885 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:27.485541 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782a53c-a70a-4f75-8ab0-a9c3d8756885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:27.534407 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-c782a53c-a70a-4f75-8ab0-a9c3d8756885 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:27.535960 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-c782a53c-a70a-4f75-8ab0-a9c3d8756885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:27.563747 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aeb744-84b6-4d22-846e-e19077b58d2f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:27.567492 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1290/2564] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:27.591251 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aeb744-84b6-4d22-846e-e19077b58d2f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:27.612861 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc2a8a-4446-4ad1-98eb-b4954bf208a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:27.614052 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc2a8a-4446-4ad1-98eb-b4954bf208a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:27.614832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc2a8a-4446-4ad1-98eb-b4954bf208a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:27.615337 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc2a8a-4446-4ad1-98eb-b4954bf208a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:27.616592 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc2a8a-4446-4ad1-98eb-b4954bf208a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:41:27.617082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc2a8a-4446-4ad1-98eb-b4954bf208a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:27.619332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc2a8a-4446-4ad1-98eb-b4954bf208a8 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:27.642895 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c782a53c-a70a-4f75-8ab0-a9c3d8756885 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:27.643795 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1275/2565] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:41:27.662639 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c782a53c-a70a-4f75-8ab0-a9c3d8756885 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:27.676332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af82f2-8afa-4c58-ae60-e939830b8ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:27.677784 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af82f2-8afa-4c58-ae60-e939830b8ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:27.678316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af82f2-8afa-4c58-ae60-e939830b8ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:27.679521 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af82f2-8afa-4c58-ae60-e939830b8ec1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:27.680759 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af82f2-8afa-4c58-ae60-e939830b8ec1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:27.685905 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af82f2-8afa-4c58-ae60-e939830b8ec1 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:27.691302 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc2a8a-4446-4ad1-98eb-b4954bf208a8 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:27.693275 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fcfc2a8a-4446-4ad1-98eb-b4954bf208a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:27.769613 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc2a8a-4446-4ad1-98eb-b4954bf208a8 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:27.784727 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1291/2566] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:41:27 2022] POST /identity/v3/projects => generated 375 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:41:27.790692 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af82f2-8afa-4c58-ae60-e939830b8ec1 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:27.814070 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e4b7d-48b1-4ddc-932a-fb1531ae747b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:27.815435 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e4b7d-48b1-4ddc-932a-fb1531ae747b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:27.815947 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e4b7d-48b1-4ddc-932a-fb1531ae747b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:27.816457 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e4b7d-48b1-4ddc-932a-fb1531ae747b admin admin] PATH_INFO: `/v3/projects/c7add0887fb448cb8f95761b746c5f00` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:27.827892 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e4b7d-48b1-4ddc-932a-fb1531ae747b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7add0887fb448cb8f95761b746c5f00)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:27.830485 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e4b7d-48b1-4ddc-932a-fb1531ae747b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:27.981280 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-994e4b7d-48b1-4ddc-932a-fb1531ae747b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7add0887fb448cb8f95761b746c5f00', 'request_id': 'req-994e4b7d-48b1-4ddc-932a-fb1531ae747b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:28.002233 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-994e4b7d-48b1-4ddc-932a-fb1531ae747b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7add0887fb448cb8f95761b746c5f00', 'request_id': 'req-994e4b7d-48b1-4ddc-932a-fb1531ae747b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:28.029073 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-994e4b7d-48b1-4ddc-932a-fb1531ae747b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7add0887fb448cb8f95761b746c5f00', 'request_id': 'req-994e4b7d-48b1-4ddc-932a-fb1531ae747b'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:28.070684 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-994e4b7d-48b1-4ddc-932a-fb1531ae747b admin admin] The token cache is being invalidate because project c7add0887fb448cb8f95761b746c5f00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:41:28.073762 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-994e4b7d-48b1-4ddc-932a-fb1531ae747b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:28.075512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-994e4b7d-48b1-4ddc-932a-fb1531ae747b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:28.079221 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af82f2-8afa-4c58-ae60-e939830b8ec1 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:28.085454 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1276/2567] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:41:27 2022] DELETE /identity/v3/projects/c7add0887fb448cb8f95761b746c5f00 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:41:28.087338 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a9a26-3391-4962-95af-1af56257b15a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:28.087736 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a9a26-3391-4962-95af-1af56257b15a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:28.088029 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a9a26-3391-4962-95af-1af56257b15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:28.090538 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:28.151683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-b77a9a26-3391-4962-95af-1af56257b15a None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:28.156554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b77a9a26-3391-4962-95af-1af56257b15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:28.288066 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad265797-59d5-41c2-9735-332764f2d1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:28.289260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad265797-59d5-41c2-9735-332764f2d1d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:28.289894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad265797-59d5-41c2-9735-332764f2d1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:28.291167 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad265797-59d5-41c2-9735-332764f2d1d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:28.292324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad265797-59d5-41c2-9735-332764f2d1d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:28.295615 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad265797-59d5-41c2-9735-332764f2d1d9 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:28.348887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1292/2568] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:41:27 2022] POST /identity/v3/users => generated 288 bytes in 563 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:28.350427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad265797-59d5-41c2-9735-332764f2d1d9 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:28.363124 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-ad265797-59d5-41c2-9735-332764f2d1d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:28.386656 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1277/2569] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:41:28.388483 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b96d2-7fc2-4cd5-b2da-e8ea65ce1c74 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:28.388967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b96d2-7fc2-4cd5-b2da-e8ea65ce1c74 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:28.389798 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b96d2-7fc2-4cd5-b2da-e8ea65ce1c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:28.400659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:28.407209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad265797-59d5-41c2-9735-332764f2d1d9 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:28.417976 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1f9b96d2-7fc2-4cd5-b2da-e8ea65ce1c74 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:28.420052 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1f9b96d2-7fc2-4cd5-b2da-e8ea65ce1c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:28.431995 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1278/2570] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:28.435149 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f47b8-554d-43e7-a98c-81ec444becc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:28.436087 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f47b8-554d-43e7-a98c-81ec444becc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:28.436508 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f47b8-554d-43e7-a98c-81ec444becc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:28.436900 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f47b8-554d-43e7-a98c-81ec444becc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:28.437445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b96d2-7fc2-4cd5-b2da-e8ea65ce1c74 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:28.437922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f47b8-554d-43e7-a98c-81ec444becc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:28.442665 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f47b8-554d-43e7-a98c-81ec444becc6 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:28.453328 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b96d2-7fc2-4cd5-b2da-e8ea65ce1c74 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:28.460375 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2693351-7c37-4b6c-8c38-fd4c8e4e37ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:28.462108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2693351-7c37-4b6c-8c38-fd4c8e4e37ce admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:28.462589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2693351-7c37-4b6c-8c38-fd4c8e4e37ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:28.463034 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2693351-7c37-4b6c-8c38-fd4c8e4e37ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:28.464019 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2693351-7c37-4b6c-8c38-fd4c8e4e37ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:41:28.464639 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2693351-7c37-4b6c-8c38-fd4c8e4e37ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:28.466503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2693351-7c37-4b6c-8c38-fd4c8e4e37ce admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:28.489833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1279/2571] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:41:28.492901 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2693351-7c37-4b6c-8c38-fd4c8e4e37ce admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:28.493510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1293/2572] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:28.494592 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f47b8-554d-43e7-a98c-81ec444becc6 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:28.507858 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2693351-7c37-4b6c-8c38-fd4c8e4e37ce admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:28.514208 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e5fbe-d132-4b17-919d-5beee9a44b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:28.515249 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e5fbe-d132-4b17-919d-5beee9a44b4c admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:28.515707 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e5fbe-d132-4b17-919d-5beee9a44b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:28.516265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e5fbe-d132-4b17-919d-5beee9a44b4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:28.518013 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e5fbe-d132-4b17-919d-5beee9a44b4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:28.520305 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e5fbe-d132-4b17-919d-5beee9a44b4c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:28.542728 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f47b8-554d-43e7-a98c-81ec444becc6 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:28.566732 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1280/2573] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:41:28 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:28.568603 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e5fbe-d132-4b17-919d-5beee9a44b4c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:28.570375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f80cb6-3211-4d4f-9e73-c19dc37e67ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:28.571440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f80cb6-3211-4d4f-9e73-c19dc37e67ed admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:28.571847 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f80cb6-3211-4d4f-9e73-c19dc37e67ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:28.572276 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f80cb6-3211-4d4f-9e73-c19dc37e67ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:28.573593 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f80cb6-3211-4d4f-9e73-c19dc37e67ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:28.586053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f80cb6-3211-4d4f-9e73-c19dc37e67ed admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:28.614461 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1294/2574] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:28.618870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f80cb6-3211-4d4f-9e73-c19dc37e67ed admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:28.637693 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f80cb6-3211-4d4f-9e73-c19dc37e67ed admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:28.717709 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e5fbe-d132-4b17-919d-5beee9a44b4c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:28.736910 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1b58b-f27c-49e9-afc1-846e8562edea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:28.744809 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1b58b-f27c-49e9-afc1-846e8562edea admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:28.746811 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1b58b-f27c-49e9-afc1-846e8562edea admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:28.748260 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1b58b-f27c-49e9-afc1-846e8562edea admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:28.753064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1b58b-f27c-49e9-afc1-846e8562edea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:28.761365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1b58b-f27c-49e9-afc1-846e8562edea admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:28.781210 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207c8c6-53fc-45cf-84cb-0110fc665c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:28.782439 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207c8c6-53fc-45cf-84cb-0110fc665c9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:28.784390 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207c8c6-53fc-45cf-84cb-0110fc665c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:28.784390 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207c8c6-53fc-45cf-84cb-0110fc665c9e admin admin] PATH_INFO: `/v3/projects/83a70bfe861e4ecfae1ddcff044e0582/users/4d99ef97227f40298df77e8c87e4add4/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:28.834991 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1281/2575] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:41:28 2022] POST /identity/v3/users => generated 287 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:28.838650 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207c8c6-53fc-45cf-84cb-0110fc665c9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=83a70bfe861e4ecfae1ddcff044e0582, user_id=4d99ef97227f40298df77e8c87e4add4, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:28.842764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207c8c6-53fc-45cf-84cb-0110fc665c9e admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:28.843265 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1b58b-f27c-49e9-afc1-846e8562edea admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:28.859689 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1b58b-f27c-49e9-afc1-846e8562edea admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:28.875994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1295/2576] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:28 2022] PUT /identity/v3/projects/83a70bfe861e4ecfae1ddcff044e0582/users/4d99ef97227f40298df77e8c87e4add4/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:28.887352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c34d0-8796-4781-8008-1c328b938dfb None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:28.887819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c34d0-8796-4781-8008-1c328b938dfb None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:28.888285 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c34d0-8796-4781-8008-1c328b938dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:28.950770 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-f64c34d0-8796-4781-8008-1c328b938dfb None None] MFA Rules not processed for user `4d99ef97227f40298df77e8c87e4add4`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:28.953782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-f64c34d0-8796-4781-8008-1c328b938dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:29.026517 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c9809-d4e5-4c67-8631-7c3152c0b7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:29.027593 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c9809-d4e5-4c67-8631-7c3152c0b7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:29.028074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c9809-d4e5-4c67-8631-7c3152c0b7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:29.028681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c9809-d4e5-4c67-8631-7c3152c0b7d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:29.029674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c9809-d4e5-4c67-8631-7c3152c0b7d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:29.031734 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c9809-d4e5-4c67-8631-7c3152c0b7d9 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:29.067444 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1282/2577] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:29.069066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c9809-d4e5-4c67-8631-7c3152c0b7d9 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:29.102033 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:29.102656 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c9809-d4e5-4c67-8631-7c3152c0b7d9 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:29.122500 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af3bf7-9352-4edc-8ee8-5a21ef9ebf49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:29.123827 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af3bf7-9352-4edc-8ee8-5a21ef9ebf49 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:29.124427 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af3bf7-9352-4edc-8ee8-5a21ef9ebf49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:29.137568 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af3bf7-9352-4edc-8ee8-5a21ef9ebf49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:29.138713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af3bf7-9352-4edc-8ee8-5a21ef9ebf49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:29.149794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af3bf7-9352-4edc-8ee8-5a21ef9ebf49 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:29.171523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1296/2578] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:41:29.177816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1283/2579] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:29.192251 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c34d0-8796-4781-8008-1c328b938dfb None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:29.192511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af3bf7-9352-4edc-8ee8-5a21ef9ebf49 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:29.206983 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-66af3bf7-9352-4edc-8ee8-5a21ef9ebf49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:29.234659 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c34d0-8796-4781-8008-1c328b938dfb None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:29.251418 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571be061-7729-4ef7-b916-74409dd58ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:29.254253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571be061-7729-4ef7-b916-74409dd58ae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:29.254918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571be061-7729-4ef7-b916-74409dd58ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:29.255571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571be061-7729-4ef7-b916-74409dd58ae0 admin admin] PATH_INFO: `/v3/projects/5a40e370a76f4649a03a5cd2e0bf8abd/users/f1cb09e72e6b45308ca64d5d2140d17f/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:29.331492 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571be061-7729-4ef7-b916-74409dd58ae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a40e370a76f4649a03a5cd2e0bf8abd, user_id=f1cb09e72e6b45308ca64d5d2140d17f, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:29.349794 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571be061-7729-4ef7-b916-74409dd58ae0 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:29.413253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:29.418671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1297/2580] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:29 2022] PUT /identity/v3/projects/5a40e370a76f4649a03a5cd2e0bf8abd/users/f1cb09e72e6b45308ca64d5d2140d17f/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:29.449559 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba28291-2a01-46a2-b9b7-511870b0a864 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:29.450253 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba28291-2a01-46a2-b9b7-511870b0a864 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:29.450706 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba28291-2a01-46a2-b9b7-511870b0a864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:29.473581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af3bf7-9352-4edc-8ee8-5a21ef9ebf49 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:29.530676 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-5ba28291-2a01-46a2-b9b7-511870b0a864 None None] MFA Rules not processed for user `f1cb09e72e6b45308ca64d5d2140d17f`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:29.532267 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-5ba28291-2a01-46a2-b9b7-511870b0a864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:29.674663 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8dd62-0d0e-45e5-9f41-70239593c604 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327b347d6295422c99ce7b44eab00467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:29.675429 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8dd62-0d0e-45e5-9f41-70239593c604 service glance] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:29.675648 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8dd62-0d0e-45e5-9f41-70239593c604 service glance] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:29.675933 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8dd62-0d0e-45e5-9f41-70239593c604 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:29.729206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1298/2581] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:29.740348 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba28291-2a01-46a2-b9b7-511870b0a864 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:29.741850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-5ba28291-2a01-46a2-b9b7-511870b0a864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:29.863252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8dd62-0d0e-45e5-9f41-70239593c604 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:29.866008 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8dd62-0d0e-45e5-9f41-70239593c604 service glance] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:29.897788 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1284/2582] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 706 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:29.971385 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba28291-2a01-46a2-b9b7-511870b0a864 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:29.991786 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30cf52-ca43-4c04-83cb-91910c9edf99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:29.993201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30cf52-ca43-4c04-83cb-91910c9edf99 service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:29.993622 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30cf52-ca43-4c04-83cb-91910c9edf99 service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:29.994021 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30cf52-ca43-4c04-83cb-91910c9edf99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:30.026880 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30cf52-ca43-4c04-83cb-91910c9edf99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:30.029916 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30cf52-ca43-4c04-83cb-91910c9edf99 service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:30.045709 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1299/2583] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:30.772550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8dd62-0d0e-45e5-9f41-70239593c604 service glance] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:30.971126 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8dd62-0d0e-45e5-9f41-70239593c604 service glance] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:30.985667 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f4c1b-d141-4654-a016-753e8ee38dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:30.988066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f4c1b-d141-4654-a016-753e8ee38dff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:30.988794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f4c1b-d141-4654-a016-753e8ee38dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:30.990016 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f4c1b-d141-4654-a016-753e8ee38dff admin admin] PATH_INFO: `/v3/users/f1cb09e72e6b45308ca64d5d2140d17f` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:30.992318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f4c1b-d141-4654-a016-753e8ee38dff admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1cb09e72e6b45308ca64d5d2140d17f)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:30.995157 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f4c1b-d141-4654-a016-753e8ee38dff admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:31.127836 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f06f4c1b-d141-4654-a016-753e8ee38dff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1cb09e72e6b45308ca64d5d2140d17f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:31.143589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:31.179674 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f06f4c1b-d141-4654-a016-753e8ee38dff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1cb09e72e6b45308ca64d5d2140d17f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:31.181031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f06f4c1b-d141-4654-a016-753e8ee38dff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1cb09e72e6b45308ca64d5d2140d17f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:31.216857 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f06f4c1b-d141-4654-a016-753e8ee38dff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1cb09e72e6b45308ca64d5d2140d17f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:31.256559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f06f4c1b-d141-4654-a016-753e8ee38dff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1cb09e72e6b45308ca64d5d2140d17f'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:31.283678 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407866b-d1c1-4a80-8961-64a2dd2de27c None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:31.285718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407866b-d1c1-4a80-8961-64a2dd2de27c None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:31.286238 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1285/2584] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:41:30 2022] DELETE /identity/v3/users/f1cb09e72e6b45308ca64d5d2140d17f => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:41:31.287053 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407866b-d1c1-4a80-8961-64a2dd2de27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:31.349509 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-5407866b-d1c1-4a80-8961-64a2dd2de27c None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:31.351227 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-5407866b-d1c1-4a80-8961-64a2dd2de27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:31.459762 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:31.531206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1300/2585] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:31.542764 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f4c1b-d141-4654-a016-753e8ee38dff admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:31.544306 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-f06f4c1b-d141-4654-a016-753e8ee38dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:31.715759 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f4c1b-d141-4654-a016-753e8ee38dff admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:31.726288 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170f7ce-2d2f-4ee0-b1cf-c4d5107fc624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:31.727654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170f7ce-2d2f-4ee0-b1cf-c4d5107fc624 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:31.728266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170f7ce-2d2f-4ee0-b1cf-c4d5107fc624 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:31.728892 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170f7ce-2d2f-4ee0-b1cf-c4d5107fc624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:31.745181 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170f7ce-2d2f-4ee0-b1cf-c4d5107fc624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:31.747179 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170f7ce-2d2f-4ee0-b1cf-c4d5107fc624 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:31.766624 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1286/2586] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:41:32.588496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407866b-d1c1-4a80-8961-64a2dd2de27c None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:32.590671 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-5407866b-d1c1-4a80-8961-64a2dd2de27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:32.612789 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407866b-d1c1-4a80-8961-64a2dd2de27c None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:32.618282 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46daae10-30c5-4b3f-957e-6bb5b4c54202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:32.619620 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46daae10-30c5-4b3f-957e-6bb5b4c54202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:32.620279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46daae10-30c5-4b3f-957e-6bb5b4c54202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:32.620932 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46daae10-30c5-4b3f-957e-6bb5b4c54202 admin admin] PATH_INFO: `/v3/projects/5a40e370a76f4649a03a5cd2e0bf8abd` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:32.623154 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46daae10-30c5-4b3f-957e-6bb5b4c54202 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a40e370a76f4649a03a5cd2e0bf8abd)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:32.625522 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46daae10-30c5-4b3f-957e-6bb5b4c54202 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:32.714796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-46daae10-30c5-4b3f-957e-6bb5b4c54202 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a40e370a76f4649a03a5cd2e0bf8abd', 'request_id': 'req-46daae10-30c5-4b3f-957e-6bb5b4c54202'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:32.728891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-46daae10-30c5-4b3f-957e-6bb5b4c54202 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a40e370a76f4649a03a5cd2e0bf8abd', 'request_id': 'req-46daae10-30c5-4b3f-957e-6bb5b4c54202'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:32.752087 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-46daae10-30c5-4b3f-957e-6bb5b4c54202 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a40e370a76f4649a03a5cd2e0bf8abd', 'request_id': 'req-46daae10-30c5-4b3f-957e-6bb5b4c54202'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:32.775756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-46daae10-30c5-4b3f-957e-6bb5b4c54202 admin admin] The token cache is being invalidate because project 5a40e370a76f4649a03a5cd2e0bf8abd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:41:32.776310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-46daae10-30c5-4b3f-957e-6bb5b4c54202 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:32.777428 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-46daae10-30c5-4b3f-957e-6bb5b4c54202 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:32.782233 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1301/2587] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:41:32 2022] DELETE /identity/v3/projects/5a40e370a76f4649a03a5cd2e0bf8abd => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:32.790052 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b343a2-034a-4a5c-9be5-7de30e246609 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:32.790545 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b343a2-034a-4a5c-9be5-7de30e246609 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:32.790654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b343a2-034a-4a5c-9be5-7de30e246609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:32.833367 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-42b343a2-034a-4a5c-9be5-7de30e246609 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:32.834085 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-42b343a2-034a-4a5c-9be5-7de30e246609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:33.023800 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1287/2588] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:33.036318 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b50b0d-865b-4785-b590-9f39a2e84ffe None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:33.037610 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b50b0d-865b-4785-b590-9f39a2e84ffe None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:33.038056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b50b0d-865b-4785-b590-9f39a2e84ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:33.078333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-94b50b0d-865b-4785-b590-9f39a2e84ffe None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:33.082961 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-94b50b0d-865b-4785-b590-9f39a2e84ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:33.110040 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1302/2589] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:33.116776 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b343a2-034a-4a5c-9be5-7de30e246609 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:33.136047 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b343a2-034a-4a5c-9be5-7de30e246609 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:33.154491 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b161b9f-6671-4849-aa09-201855546d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:33.156204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b161b9f-6671-4849-aa09-201855546d79 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:33.156979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b161b9f-6671-4849-aa09-201855546d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:33.158910 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b161b9f-6671-4849-aa09-201855546d79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:33.160559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b161b9f-6671-4849-aa09-201855546d79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:41:33.161064 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b161b9f-6671-4849-aa09-201855546d79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:33.168646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b161b9f-6671-4849-aa09-201855546d79 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:33.219778 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1288/2590] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:41:33.226794 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b50b0d-865b-4785-b590-9f39a2e84ffe None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:33.244056 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b50b0d-865b-4785-b590-9f39a2e84ffe None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:33.255175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49296ff1-08f0-4271-bc95-a1e99bc6db4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:33.255911 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49296ff1-08f0-4271-bc95-a1e99bc6db4e admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:33.256203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49296ff1-08f0-4271-bc95-a1e99bc6db4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:33.256449 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49296ff1-08f0-4271-bc95-a1e99bc6db4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:33.269976 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49296ff1-08f0-4271-bc95-a1e99bc6db4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:33.274224 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49296ff1-08f0-4271-bc95-a1e99bc6db4e admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:33.349224 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1303/2591] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:41:33 2022] POST /identity/v3/projects => generated 405 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:33.352709 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b161b9f-6671-4849-aa09-201855546d79 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:33.377319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b161b9f-6671-4849-aa09-201855546d79 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:33.493081 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:33.506253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad86dd-1509-48a9-9d3e-dc2146a97e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:33.508162 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad86dd-1509-48a9-9d3e-dc2146a97e48 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:33.508496 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad86dd-1509-48a9-9d3e-dc2146a97e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:33.508908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad86dd-1509-48a9-9d3e-dc2146a97e48 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:33.510147 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad86dd-1509-48a9-9d3e-dc2146a97e48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:33.512384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad86dd-1509-48a9-9d3e-dc2146a97e48 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:33.584804 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1289/2592] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:41:33 2022] POST /identity/v3/users => generated 303 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:33.586140 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49296ff1-08f0-4271-bc95-a1e99bc6db4e admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:33.610261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49296ff1-08f0-4271-bc95-a1e99bc6db4e admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:33.622370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac86fa-0ef4-4768-9464-3c5b39aadfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:33.624717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac86fa-0ef4-4768-9464-3c5b39aadfc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:33.626069 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac86fa-0ef4-4768-9464-3c5b39aadfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:33.627615 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac86fa-0ef4-4768-9464-3c5b39aadfc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:33.641207 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac86fa-0ef4-4768-9464-3c5b39aadfc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:33.651704 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac86fa-0ef4-4768-9464-3c5b39aadfc2 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:33.705295 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad86dd-1509-48a9-9d3e-dc2146a97e48 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:33.715782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1304/2593] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:33.725919 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad86dd-1509-48a9-9d3e-dc2146a97e48 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:33.733386 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db7e87-be75-43e5-a251-17cd92d953de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:33.735079 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db7e87-be75-43e5-a251-17cd92d953de admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:33.744655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db7e87-be75-43e5-a251-17cd92d953de admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:33.745365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db7e87-be75-43e5-a251-17cd92d953de admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:33.746923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12db7e87-be75-43e5-a251-17cd92d953de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:33.750137 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12db7e87-be75-43e5-a251-17cd92d953de admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:33.791069 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac86fa-0ef4-4768-9464-3c5b39aadfc2 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:33.792879 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1290/2594] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:33.811491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac86fa-0ef4-4768-9464-3c5b39aadfc2 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:33.831626 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953f2e6-0e1b-46a6-9c68-508415093dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:33.833391 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953f2e6-0e1b-46a6-9c68-508415093dd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:33.833878 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953f2e6-0e1b-46a6-9c68-508415093dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:33.834385 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953f2e6-0e1b-46a6-9c68-508415093dd4 admin admin] PATH_INFO: `/v3/projects/aefa82123c91486bb1c42e5ff09e8bdf/users/f8640202d9fe47a085730c3d2d235451/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:33.866442 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d953f2e6-0e1b-46a6-9c68-508415093dd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aefa82123c91486bb1c42e5ff09e8bdf, user_id=f8640202d9fe47a085730c3d2d235451, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:33.871985 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d953f2e6-0e1b-46a6-9c68-508415093dd4 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:33.910033 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1305/2595] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:33 2022] PUT /identity/v3/projects/aefa82123c91486bb1c42e5ff09e8bdf/users/f8640202d9fe47a085730c3d2d235451/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:33.915627 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75df27f-ec29-49f9-a635-306a014592ad None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:33.915627 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75df27f-ec29-49f9-a635-306a014592ad None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:33.916027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75df27f-ec29-49f9-a635-306a014592ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:33.937220 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-b75df27f-ec29-49f9-a635-306a014592ad None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:33.938769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b75df27f-ec29-49f9-a635-306a014592ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:34.019162 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1291/2596] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:41:34.032086 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953f2e6-0e1b-46a6-9c68-508415093dd4 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:34.036124 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-d953f2e6-0e1b-46a6-9c68-508415093dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:34.138792 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953f2e6-0e1b-46a6-9c68-508415093dd4 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:34.144946 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49027f-fddd-45d3-984b-eee246d1fedd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:34.145964 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49027f-fddd-45d3-984b-eee246d1fedd service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:34.146246 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49027f-fddd-45d3-984b-eee246d1fedd service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:34.146532 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49027f-fddd-45d3-984b-eee246d1fedd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:34.164835 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d49027f-fddd-45d3-984b-eee246d1fedd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:34.172491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d49027f-fddd-45d3-984b-eee246d1fedd service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:34.174970 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:34.182919 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1306/2597] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:34.473602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076fb68-bca5-4508-a835-6b36097715cd None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:34.475027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076fb68-bca5-4508-a835-6b36097715cd None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:34.475418 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076fb68-bca5-4508-a835-6b36097715cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:34.521610 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-b076fb68-bca5-4508-a835-6b36097715cd None None] MFA Rules not processed for user `cfc2f31f654e486d8505d80aad17d17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:34.523510 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b076fb68-bca5-4508-a835-6b36097715cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:34.612165 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1292/2598] 10.222.0.40 () {58 vars in 967 bytes} [Thu Jun 16 17:41:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:34.630855 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49027f-fddd-45d3-984b-eee246d1fedd service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:34.632680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-3d49027f-fddd-45d3-984b-eee246d1fedd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:34.730645 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49027f-fddd-45d3-984b-eee246d1fedd service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:34.748372 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0714c7-fb98-4f42-bc8b-f9ed021e0508 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a37d1201d54afdad9b228ef4f2d787', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:34.749352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0714c7-fb98-4f42-bc8b-f9ed021e0508 service swift] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:34.749737 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0714c7-fb98-4f42-bc8b-f9ed021e0508 service swift] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:34.750311 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0714c7-fb98-4f42-bc8b-f9ed021e0508 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:34.761205 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0714c7-fb98-4f42-bc8b-f9ed021e0508 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:34.763189 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0714c7-fb98-4f42-bc8b-f9ed021e0508 service swift] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:34.777175 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1307/2599] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 17:41:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:41:35.412290 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923b7cb-3508-4624-9aae-8eb04efe5efe None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:35.413886 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923b7cb-3508-4624-9aae-8eb04efe5efe None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:35.414503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923b7cb-3508-4624-9aae-8eb04efe5efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:35.478891 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4923b7cb-3508-4624-9aae-8eb04efe5efe None None] MFA Rules not processed for user `f8640202d9fe47a085730c3d2d235451`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:35.481103 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4923b7cb-3508-4624-9aae-8eb04efe5efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:35.603589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1293/2600] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:35 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:35.609512 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0714c7-fb98-4f42-bc8b-f9ed021e0508 service swift] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:35.628038 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0714c7-fb98-4f42-bc8b-f9ed021e0508 service swift] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:35.633656 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f9df5-64da-4ea2-8384-1b52178530e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:35.634345 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f9df5-64da-4ea2-8384-1b52178530e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:35.634627 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f9df5-64da-4ea2-8384-1b52178530e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:35.634910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f9df5-64da-4ea2-8384-1b52178530e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:35.635630 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f9df5-64da-4ea2-8384-1b52178530e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:35.637711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f9df5-64da-4ea2-8384-1b52178530e7 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:35.680459 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1308/2601] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:41:35 2022] POST /identity/v3/projects => generated 405 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:35.686555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923b7cb-3508-4624-9aae-8eb04efe5efe None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:35.706450 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923b7cb-3508-4624-9aae-8eb04efe5efe None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:35.723624 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:35.793132 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103a5cc-69ef-436f-b74c-db3f217bdc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:35.794096 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103a5cc-69ef-436f-b74c-db3f217bdc95 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:35.794830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103a5cc-69ef-436f-b74c-db3f217bdc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:35.795263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103a5cc-69ef-436f-b74c-db3f217bdc95 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:35.797472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7103a5cc-69ef-436f-b74c-db3f217bdc95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:35.800155 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7103a5cc-69ef-436f-b74c-db3f217bdc95 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:35.835608 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1294/2602] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:41:35 2022] POST /identity/v3/users => generated 309 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:35.839460 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f9df5-64da-4ea2-8384-1b52178530e7 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:35.852997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f9df5-64da-4ea2-8384-1b52178530e7 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:35.861182 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e132f74-233e-4cc9-b7ac-b3a330caf9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:35.861934 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e132f74-233e-4cc9-b7ac-b3a330caf9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:35.862288 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e132f74-233e-4cc9-b7ac-b3a330caf9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:35.862631 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e132f74-233e-4cc9-b7ac-b3a330caf9f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:35.863473 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e132f74-233e-4cc9-b7ac-b3a330caf9f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:35.865104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e132f74-233e-4cc9-b7ac-b3a330caf9f9 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:35.879771 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1309/2603] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:35.885249 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103a5cc-69ef-436f-b74c-db3f217bdc95 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:35.900848 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103a5cc-69ef-436f-b74c-db3f217bdc95 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:35.910292 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605737b-9b6f-4aa7-8ff0-5cda744c1bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:35.911968 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605737b-9b6f-4aa7-8ff0-5cda744c1bae admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:35.912419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605737b-9b6f-4aa7-8ff0-5cda744c1bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:35.913190 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605737b-9b6f-4aa7-8ff0-5cda744c1bae admin admin] PATH_INFO: `/v3/projects/1e3f3de73a414f23a8db15c5d8353c2f/users/881bfe6e964443ae9e24eed45f952241/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:35.943594 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6605737b-9b6f-4aa7-8ff0-5cda744c1bae admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e3f3de73a414f23a8db15c5d8353c2f, user_id=881bfe6e964443ae9e24eed45f952241, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:35.949068 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6605737b-9b6f-4aa7-8ff0-5cda744c1bae admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:35.977743 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1295/2604] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:35 2022] PUT /identity/v3/projects/1e3f3de73a414f23a8db15c5d8353c2f/users/881bfe6e964443ae9e24eed45f952241/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:35.982023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e132f74-233e-4cc9-b7ac-b3a330caf9f9 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:35.999272 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e132f74-233e-4cc9-b7ac-b3a330caf9f9 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:36.097435 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215735bb-a4c5-4089-ae7b-3ce5bfe8bbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:36.098615 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215735bb-a4c5-4089-ae7b-3ce5bfe8bbfc admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:36.099567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215735bb-a4c5-4089-ae7b-3ce5bfe8bbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:36.099696 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215735bb-a4c5-4089-ae7b-3ce5bfe8bbfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:36.101313 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215735bb-a4c5-4089-ae7b-3ce5bfe8bbfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:36.104558 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215735bb-a4c5-4089-ae7b-3ce5bfe8bbfc admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:36.125849 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1310/2605] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:36.132008 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605737b-9b6f-4aa7-8ff0-5cda744c1bae admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:36.151883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605737b-9b6f-4aa7-8ff0-5cda744c1bae admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:36.159303 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fba85f-53ad-40f1-86c4-f051ad13c29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:36.160223 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fba85f-53ad-40f1-86c4-f051ad13c29a admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:36.160507 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fba85f-53ad-40f1-86c4-f051ad13c29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:36.160789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fba85f-53ad-40f1-86c4-f051ad13c29a admin admin] PATH_INFO: `/v3/projects/1e3f3de73a414f23a8db15c5d8353c2f/users/881bfe6e964443ae9e24eed45f952241/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:36.162894 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fba85f-53ad-40f1-86c4-f051ad13c29a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e3f3de73a414f23a8db15c5d8353c2f, user_id=881bfe6e964443ae9e24eed45f952241, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:36.165589 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fba85f-53ad-40f1-86c4-f051ad13c29a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:36.177438 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1296/2606] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:36 2022] PUT /identity/v3/projects/1e3f3de73a414f23a8db15c5d8353c2f/users/881bfe6e964443ae9e24eed45f952241/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:37.178358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd47f9-f2a8-4e12-bd77-6499d83b9c7f None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:37.179244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd47f9-f2a8-4e12-bd77-6499d83b9c7f None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:37.179420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd47f9-f2a8-4e12-bd77-6499d83b9c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:37.234035 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-b8cd47f9-f2a8-4e12-bd77-6499d83b9c7f None None] MFA Rules not processed for user `881bfe6e964443ae9e24eed45f952241`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:37.239305 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b8cd47f9-f2a8-4e12-bd77-6499d83b9c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:37.319271 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:37.366839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1311/2607] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:37 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:37.937679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fba85f-53ad-40f1-86c4-f051ad13c29a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:37.941841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-55fba85f-53ad-40f1-86c4-f051ad13c29a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:38.054045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fba85f-53ad-40f1-86c4-f051ad13c29a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:38.061531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d94cca-c510-45d7-8baa-df9512082978 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75f0586a8c1941448d750918183f6dc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:38.062426 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d94cca-c510-45d7-8baa-df9512082978 service placement] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:38.062917 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d94cca-c510-45d7-8baa-df9512082978 service placement] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:38.063431 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d94cca-c510-45d7-8baa-df9512082978 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:38.070550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d3d94cca-c510-45d7-8baa-df9512082978 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:38.091096 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d94cca-c510-45d7-8baa-df9512082978 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:38.093082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d94cca-c510-45d7-8baa-df9512082978 service placement] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:38.113339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1297/2608] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:38.746337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd47f9-f2a8-4e12-bd77-6499d83b9c7f None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:38.841020 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd47f9-f2a8-4e12-bd77-6499d83b9c7f None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:38.849215 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae96d2e4-81ac-4f20-bde4-7384a3d8191b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:38.850722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96d2e4-81ac-4f20-bde4-7384a3d8191b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:38.850722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96d2e4-81ac-4f20-bde4-7384a3d8191b admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:38.851369 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96d2e4-81ac-4f20-bde4-7384a3d8191b admin admin] PATH_INFO: `/v3/users/f8640202d9fe47a085730c3d2d235451` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:38.852261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae96d2e4-81ac-4f20-bde4-7384a3d8191b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8640202d9fe47a085730c3d2d235451)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:38.854486 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae96d2e4-81ac-4f20-bde4-7384a3d8191b admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:38.916414 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-ae96d2e4-81ac-4f20-bde4-7384a3d8191b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8640202d9fe47a085730c3d2d235451'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:38.925039 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-ae96d2e4-81ac-4f20-bde4-7384a3d8191b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8640202d9fe47a085730c3d2d235451'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:38.925695 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-ae96d2e4-81ac-4f20-bde4-7384a3d8191b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8640202d9fe47a085730c3d2d235451'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:38.933952 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-ae96d2e4-81ac-4f20-bde4-7384a3d8191b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8640202d9fe47a085730c3d2d235451'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:38.953782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-ae96d2e4-81ac-4f20-bde4-7384a3d8191b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8640202d9fe47a085730c3d2d235451'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:38.961463 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1312/2609] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:41:38 2022] DELETE /identity/v3/users/f8640202d9fe47a085730c3d2d235451 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:39.192520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d94cca-c510-45d7-8baa-df9512082978 service placement] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:39.211987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d94cca-c510-45d7-8baa-df9512082978 service placement] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:39.289983 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:39.302099 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886fb85e-0663-4133-954f-e8c39e7f2add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:39.302881 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fb85e-0663-4133-954f-e8c39e7f2add admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:39.303194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fb85e-0663-4133-954f-e8c39e7f2add admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:39.303578 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fb85e-0663-4133-954f-e8c39e7f2add admin admin] PATH_INFO: `/v3/projects/aefa82123c91486bb1c42e5ff09e8bdf` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:39.304866 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886fb85e-0663-4133-954f-e8c39e7f2add admin admin] RBAC: Authorizing `identity:delete_project(project_id=aefa82123c91486bb1c42e5ff09e8bdf)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:39.307027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886fb85e-0663-4133-954f-e8c39e7f2add admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:39.364342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-886fb85e-0663-4133-954f-e8c39e7f2add admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aefa82123c91486bb1c42e5ff09e8bdf', 'request_id': 'req-886fb85e-0663-4133-954f-e8c39e7f2add'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:39.374072 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-886fb85e-0663-4133-954f-e8c39e7f2add admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aefa82123c91486bb1c42e5ff09e8bdf', 'request_id': 'req-886fb85e-0663-4133-954f-e8c39e7f2add'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:39.392106 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-886fb85e-0663-4133-954f-e8c39e7f2add admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aefa82123c91486bb1c42e5ff09e8bdf', 'request_id': 'req-886fb85e-0663-4133-954f-e8c39e7f2add'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:39.411990 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-886fb85e-0663-4133-954f-e8c39e7f2add admin admin] The token cache is being invalidate because project aefa82123c91486bb1c42e5ff09e8bdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:41:39.412475 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-886fb85e-0663-4133-954f-e8c39e7f2add admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:39.413696 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-886fb85e-0663-4133-954f-e8c39e7f2add admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:39.415532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1298/2610] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:41:39 2022] DELETE /identity/v3/projects/aefa82123c91486bb1c42e5ff09e8bdf => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:39.421132 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae96d2e4-81ac-4f20-bde4-7384a3d8191b admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:39.423234 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-ae96d2e4-81ac-4f20-bde4-7384a3d8191b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:39.443283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae96d2e4-81ac-4f20-bde4-7384a3d8191b admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:39.518778 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:39.530883 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e7c12-587b-465a-af3c-689f33164b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:39.532110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e7c12-587b-465a-af3c-689f33164b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:39.532690 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e7c12-587b-465a-af3c-689f33164b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:39.533894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e7c12-587b-465a-af3c-689f33164b53 admin admin] PATH_INFO: `/v3/users/881bfe6e964443ae9e24eed45f952241` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:39.535959 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e7c12-587b-465a-af3c-689f33164b53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=881bfe6e964443ae9e24eed45f952241)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:39.538238 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e7c12-587b-465a-af3c-689f33164b53 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:39.605552 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d06e7c12-587b-465a-af3c-689f33164b53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '881bfe6e964443ae9e24eed45f952241'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:39.615634 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d06e7c12-587b-465a-af3c-689f33164b53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '881bfe6e964443ae9e24eed45f952241'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:39.616385 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d06e7c12-587b-465a-af3c-689f33164b53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '881bfe6e964443ae9e24eed45f952241'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:39.625049 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d06e7c12-587b-465a-af3c-689f33164b53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '881bfe6e964443ae9e24eed45f952241'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:39.647824 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d06e7c12-587b-465a-af3c-689f33164b53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '881bfe6e964443ae9e24eed45f952241'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:39.656904 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1313/2611] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:41:39 2022] DELETE /identity/v3/users/881bfe6e964443ae9e24eed45f952241 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:39.896278 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886fb85e-0663-4133-954f-e8c39e7f2add admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:39.897749 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-886fb85e-0663-4133-954f-e8c39e7f2add admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:39.931153 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886fb85e-0663-4133-954f-e8c39e7f2add admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:40.043558 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b999d-3c35-4908-8d3c-5e93c6fc3551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:40.044944 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b999d-3c35-4908-8d3c-5e93c6fc3551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:40.045359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b999d-3c35-4908-8d3c-5e93c6fc3551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:40.045778 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b999d-3c35-4908-8d3c-5e93c6fc3551 admin admin] PATH_INFO: `/v3/projects/1e3f3de73a414f23a8db15c5d8353c2f` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:40.048711 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b999d-3c35-4908-8d3c-5e93c6fc3551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e3f3de73a414f23a8db15c5d8353c2f)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:40.052854 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b999d-3c35-4908-8d3c-5e93c6fc3551 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:40.125449 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-989b999d-3c35-4908-8d3c-5e93c6fc3551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e3f3de73a414f23a8db15c5d8353c2f', 'request_id': 'req-989b999d-3c35-4908-8d3c-5e93c6fc3551'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:40.136784 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-989b999d-3c35-4908-8d3c-5e93c6fc3551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e3f3de73a414f23a8db15c5d8353c2f', 'request_id': 'req-989b999d-3c35-4908-8d3c-5e93c6fc3551'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:40.144379 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-989b999d-3c35-4908-8d3c-5e93c6fc3551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e3f3de73a414f23a8db15c5d8353c2f', 'request_id': 'req-989b999d-3c35-4908-8d3c-5e93c6fc3551'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:40.160877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-989b999d-3c35-4908-8d3c-5e93c6fc3551 admin admin] The token cache is being invalidate because project 1e3f3de73a414f23a8db15c5d8353c2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:41:40.161462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-989b999d-3c35-4908-8d3c-5e93c6fc3551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:40.162074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-989b999d-3c35-4908-8d3c-5e93c6fc3551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:40.167377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1299/2612] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:41:39 2022] DELETE /identity/v3/projects/1e3f3de73a414f23a8db15c5d8353c2f => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:40.169818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab434960-9517-4046-b8d7-37a31a950786 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:40.170413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab434960-9517-4046-b8d7-37a31a950786 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:40.170530 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab434960-9517-4046-b8d7-37a31a950786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:40.192223 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-ab434960-9517-4046-b8d7-37a31a950786 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:40.193522 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-ab434960-9517-4046-b8d7-37a31a950786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:40.270499 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1314/2613] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:40.278325 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288b87d-4c35-469f-b6cd-491771d22408 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:40.279140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288b87d-4c35-469f-b6cd-491771d22408 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:40.279140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288b87d-4c35-469f-b6cd-491771d22408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:40.304802 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-6288b87d-4c35-469f-b6cd-491771d22408 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:40.306380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6288b87d-4c35-469f-b6cd-491771d22408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:40.315258 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1300/2614] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:40.319573 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab434960-9517-4046-b8d7-37a31a950786 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:40.329853 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab434960-9517-4046-b8d7-37a31a950786 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:40.336218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd8b34-896e-4248-b081-36b571ecc766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:40.337455 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd8b34-896e-4248-b081-36b571ecc766 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:40.337785 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd8b34-896e-4248-b081-36b571ecc766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:40.338375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd8b34-896e-4248-b081-36b571ecc766 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:40.339389 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd8b34-896e-4248-b081-36b571ecc766 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:41:40.339708 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd8b34-896e-4248-b081-36b571ecc766 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:40.341407 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd8b34-896e-4248-b081-36b571ecc766 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:40.357505 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1315/2615] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:41:40.362871 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6288b87d-4c35-469f-b6cd-491771d22408 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:40.372356 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6288b87d-4c35-469f-b6cd-491771d22408 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:40.378215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e96cc-777a-4239-b294-2ca973a61f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:40.378899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e96cc-777a-4239-b294-2ca973a61f73 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:40.379189 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e96cc-777a-4239-b294-2ca973a61f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:40.379471 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e96cc-777a-4239-b294-2ca973a61f73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:40.380213 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e96cc-777a-4239-b294-2ca973a61f73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:40.382290 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e96cc-777a-4239-b294-2ca973a61f73 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:40.386453 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:40.415512 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1301/2616] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:41:40 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:40.419802 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd8b34-896e-4248-b081-36b571ecc766 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:40.431553 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd8b34-896e-4248-b081-36b571ecc766 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:40.495520 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa89e6c-4a02-40d7-aa1b-b021c3577252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:40.496120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa89e6c-4a02-40d7-aa1b-b021c3577252 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:40.496418 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa89e6c-4a02-40d7-aa1b-b021c3577252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:40.496694 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa89e6c-4a02-40d7-aa1b-b021c3577252 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:40.497450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa89e6c-4a02-40d7-aa1b-b021c3577252 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:40.499403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa89e6c-4a02-40d7-aa1b-b021c3577252 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:40.511674 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:40.525237 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1316/2617] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:41:40 2022] POST /identity/v3/users => generated 284 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:40.533376 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e96cc-777a-4239-b294-2ca973a61f73 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:40.550439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e96cc-777a-4239-b294-2ca973a61f73 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:40.556051 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23adbfc-ed3d-4673-8794-483d1bdbb0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:40.556713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23adbfc-ed3d-4673-8794-483d1bdbb0cd admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:40.557003 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23adbfc-ed3d-4673-8794-483d1bdbb0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:40.557331 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23adbfc-ed3d-4673-8794-483d1bdbb0cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:40.558081 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23adbfc-ed3d-4673-8794-483d1bdbb0cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:40.559712 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23adbfc-ed3d-4673-8794-483d1bdbb0cd admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:40.572351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1302/2618] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:40.576922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa89e6c-4a02-40d7-aa1b-b021c3577252 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:40.589627 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa89e6c-4a02-40d7-aa1b-b021c3577252 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:40.596566 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8946a5be-736d-4144-99a7-5e44a52a1a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:40.597569 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946a5be-736d-4144-99a7-5e44a52a1a31 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:40.597761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946a5be-736d-4144-99a7-5e44a52a1a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:40.598117 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946a5be-736d-4144-99a7-5e44a52a1a31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:40.599220 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8946a5be-736d-4144-99a7-5e44a52a1a31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:40.600735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8946a5be-736d-4144-99a7-5e44a52a1a31 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:40.615666 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1317/2619] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:40.621123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23adbfc-ed3d-4673-8794-483d1bdbb0cd admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:40.635194 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23adbfc-ed3d-4673-8794-483d1bdbb0cd admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:40.640107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041c672-9b0f-4c51-889b-86c2dbc7a38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:40.641011 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041c672-9b0f-4c51-889b-86c2dbc7a38a admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:40.641334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041c672-9b0f-4c51-889b-86c2dbc7a38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:40.641616 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041c672-9b0f-4c51-889b-86c2dbc7a38a admin admin] PATH_INFO: `/v3/projects/c9d0d90eccb5403eb3f93d7b1fec273c/users/f47e890d1a5b495d8148139caefc4d11/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:40.658859 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041c672-9b0f-4c51-889b-86c2dbc7a38a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9d0d90eccb5403eb3f93d7b1fec273c, user_id=f47e890d1a5b495d8148139caefc4d11, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:40.663187 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041c672-9b0f-4c51-889b-86c2dbc7a38a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:40.691602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1303/2620] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:40 2022] PUT /identity/v3/projects/c9d0d90eccb5403eb3f93d7b1fec273c/users/f47e890d1a5b495d8148139caefc4d11/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:40.703862 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de7f90-5594-42c5-8225-0cccefc92bbe None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:40.704225 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de7f90-5594-42c5-8225-0cccefc92bbe None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:40.704396 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de7f90-5594-42c5-8225-0cccefc92bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:40.734523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-17de7f90-5594-42c5-8225-0cccefc92bbe None None] MFA Rules not processed for user `f47e890d1a5b495d8148139caefc4d11`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:40.736381 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-17de7f90-5594-42c5-8225-0cccefc92bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:40.831848 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1318/2621] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:40 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:40.838831 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041c672-9b0f-4c51-889b-86c2dbc7a38a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:40.948394 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041c672-9b0f-4c51-889b-86c2dbc7a38a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:40.954625 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24145187-273c-44c8-95c8-16c654674926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:40.955584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24145187-273c-44c8-95c8-16c654674926 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:40.955878 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24145187-273c-44c8-95c8-16c654674926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:40.956174 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24145187-273c-44c8-95c8-16c654674926 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:40.956902 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24145187-273c-44c8-95c8-16c654674926 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:40.958883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24145187-273c-44c8-95c8-16c654674926 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:40.990315 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1304/2622] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:41:40 2022] POST /identity/v3/projects => generated 369 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:40.994462 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de7f90-5594-42c5-8225-0cccefc92bbe None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:41.007693 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de7f90-5594-42c5-8225-0cccefc92bbe None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:41.074181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc19cb0-29fc-485b-ba66-2860047c5a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:41.074806 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc19cb0-29fc-485b-ba66-2860047c5a2a admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:41.075102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc19cb0-29fc-485b-ba66-2860047c5a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:41.075387 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc19cb0-29fc-485b-ba66-2860047c5a2a admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:41.076131 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc19cb0-29fc-485b-ba66-2860047c5a2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:41.078171 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc19cb0-29fc-485b-ba66-2860047c5a2a admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:41.101852 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1319/2623] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:41:40 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:41.105999 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24145187-273c-44c8-95c8-16c654674926 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:41.116912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24145187-273c-44c8-95c8-16c654674926 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:41.121648 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81651b-352d-40ea-9f17-c1387e2cee2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:41.122329 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81651b-352d-40ea-9f17-c1387e2cee2e admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:41.122617 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81651b-352d-40ea-9f17-c1387e2cee2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:41.123143 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81651b-352d-40ea-9f17-c1387e2cee2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:41.123665 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81651b-352d-40ea-9f17-c1387e2cee2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:41.125294 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81651b-352d-40ea-9f17-c1387e2cee2e admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:41.138458 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1305/2624] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:41.143990 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc19cb0-29fc-485b-ba66-2860047c5a2a admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:41.156004 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc19cb0-29fc-485b-ba66-2860047c5a2a admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:41.161547 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f6935-8a74-4343-acb3-493bda091b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:41.162453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f6935-8a74-4343-acb3-493bda091b1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:41.162773 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f6935-8a74-4343-acb3-493bda091b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:41.163095 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f6935-8a74-4343-acb3-493bda091b1b admin admin] PATH_INFO: `/v3/projects/c898da3a09c64a2c85508992f08c9f35/users/5543f8f60fe0479796349a0c20977726/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:41.179058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f6935-8a74-4343-acb3-493bda091b1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c898da3a09c64a2c85508992f08c9f35, user_id=5543f8f60fe0479796349a0c20977726, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:41.181762 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f6935-8a74-4343-acb3-493bda091b1b admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:41.196327 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1320/2625] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:41 2022] PUT /identity/v3/projects/c898da3a09c64a2c85508992f08c9f35/users/5543f8f60fe0479796349a0c20977726/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:41.200334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81651b-352d-40ea-9f17-c1387e2cee2e admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:41.211541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81651b-352d-40ea-9f17-c1387e2cee2e admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:41.275673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400dcf3-56a2-495e-b063-5723272102b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:41.276355 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400dcf3-56a2-495e-b063-5723272102b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:41.276645 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400dcf3-56a2-495e-b063-5723272102b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:41.276929 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400dcf3-56a2-495e-b063-5723272102b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:41.277704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400dcf3-56a2-495e-b063-5723272102b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:41.279319 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400dcf3-56a2-495e-b063-5723272102b6 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:41.289361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1306/2626] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:41.293544 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f6935-8a74-4343-acb3-493bda091b1b admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:41.304532 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f6935-8a74-4343-acb3-493bda091b1b admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:41.310683 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d839e-b016-48d6-aa37-4daa07e10e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:41.311591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d839e-b016-48d6-aa37-4daa07e10e0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:41.311886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d839e-b016-48d6-aa37-4daa07e10e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:41.312173 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d839e-b016-48d6-aa37-4daa07e10e0e admin admin] PATH_INFO: `/v3/projects/c898da3a09c64a2c85508992f08c9f35/users/5543f8f60fe0479796349a0c20977726/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:41.315319 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d839e-b016-48d6-aa37-4daa07e10e0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c898da3a09c64a2c85508992f08c9f35, user_id=5543f8f60fe0479796349a0c20977726, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:41.318048 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d839e-b016-48d6-aa37-4daa07e10e0e admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:41.329567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1321/2627] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:41 2022] PUT /identity/v3/projects/c898da3a09c64a2c85508992f08c9f35/users/5543f8f60fe0479796349a0c20977726/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:41.339778 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9715dac-d174-43ea-877d-4ce04593c56d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:41.340105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9715dac-d174-43ea-877d-4ce04593c56d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:41.340385 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9715dac-d174-43ea-877d-4ce04593c56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:41.369162 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e9715dac-d174-43ea-877d-4ce04593c56d None None] MFA Rules not processed for user `5543f8f60fe0479796349a0c20977726`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:41.370400 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e9715dac-d174-43ea-877d-4ce04593c56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:41.379329 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:41.444706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1307/2628] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:41 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:41.456547 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d839e-b016-48d6-aa37-4daa07e10e0e admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:41.458325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-ff0d839e-b016-48d6-aa37-4daa07e10e0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:41.490261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:41.529005 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d839e-b016-48d6-aa37-4daa07e10e0e admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:41.532842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebedb97-c149-4baa-8869-ddef4ec5823d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:41.533523 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebedb97-c149-4baa-8869-ddef4ec5823d service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:41.533823 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebedb97-c149-4baa-8869-ddef4ec5823d service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:41.534120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebedb97-c149-4baa-8869-ddef4ec5823d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:41.544742 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebedb97-c149-4baa-8869-ddef4ec5823d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:41.546235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebedb97-c149-4baa-8869-ddef4ec5823d service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:41.556526 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1322/2629] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:41 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:41.600074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9715dac-d174-43ea-877d-4ce04593c56d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:41.601552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e9715dac-d174-43ea-877d-4ce04593c56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:41.615116 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9715dac-d174-43ea-877d-4ce04593c56d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:41.618769 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0464d2-5550-4276-8b8c-1e108c95fe64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:41.619428 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0464d2-5550-4276-8b8c-1e108c95fe64 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:41.619713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0464d2-5550-4276-8b8c-1e108c95fe64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:41.619993 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0464d2-5550-4276-8b8c-1e108c95fe64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:41.680051 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0464d2-5550-4276-8b8c-1e108c95fe64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:41.681531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0464d2-5550-4276-8b8c-1e108c95fe64 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:41.691254 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1308/2630] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:41 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:45.050748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebedb97-c149-4baa-8869-ddef4ec5823d service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:45.123845 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebedb97-c149-4baa-8869-ddef4ec5823d service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:45.126576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba24a47-2c1e-4220-a460-49e0ee643968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:45.127674 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba24a47-2c1e-4220-a460-49e0ee643968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:45.127965 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba24a47-2c1e-4220-a460-49e0ee643968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:45.128246 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba24a47-2c1e-4220-a460-49e0ee643968 admin admin] PATH_INFO: `/v3/users/f47e890d1a5b495d8148139caefc4d11` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:45.129497 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba24a47-2c1e-4220-a460-49e0ee643968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f47e890d1a5b495d8148139caefc4d11)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:45.131460 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba24a47-2c1e-4220-a460-49e0ee643968 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:45.183699 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-aba24a47-2c1e-4220-a460-49e0ee643968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f47e890d1a5b495d8148139caefc4d11'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:45.191732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-aba24a47-2c1e-4220-a460-49e0ee643968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f47e890d1a5b495d8148139caefc4d11'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:45.192445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-aba24a47-2c1e-4220-a460-49e0ee643968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f47e890d1a5b495d8148139caefc4d11'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:45.199383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-aba24a47-2c1e-4220-a460-49e0ee643968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f47e890d1a5b495d8148139caefc4d11'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:45.212990 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-aba24a47-2c1e-4220-a460-49e0ee643968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f47e890d1a5b495d8148139caefc4d11'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:45.219264 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1323/2631] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:41:45 2022] DELETE /identity/v3/users/f47e890d1a5b495d8148139caefc4d11 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:45.224191 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278fa27-86d3-440c-81d3-cd92e04b1a61 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:45.224695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278fa27-86d3-440c-81d3-cd92e04b1a61 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:45.224791 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278fa27-86d3-440c-81d3-cd92e04b1a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:45.243724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-a278fa27-86d3-440c-81d3-cd92e04b1a61 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:45.244965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a278fa27-86d3-440c-81d3-cd92e04b1a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:45.314833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1309/2632] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:45.324913 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba24a47-2c1e-4220-a460-49e0ee643968 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:45.326980 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-aba24a47-2c1e-4220-a460-49e0ee643968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:45.387603 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:45.395273 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba24a47-2c1e-4220-a460-49e0ee643968 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:45.399527 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32636c0d-6235-40f0-bd17-8aa000dfc80d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:45.400203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32636c0d-6235-40f0-bd17-8aa000dfc80d service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:45.400501 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32636c0d-6235-40f0-bd17-8aa000dfc80d service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:45.400786 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32636c0d-6235-40f0-bd17-8aa000dfc80d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:45.412275 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32636c0d-6235-40f0-bd17-8aa000dfc80d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:45.413787 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32636c0d-6235-40f0-bd17-8aa000dfc80d service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:45.424932 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1324/2633] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:45.738230 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278fa27-86d3-440c-81d3-cd92e04b1a61 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:45.739667 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a278fa27-86d3-440c-81d3-cd92e04b1a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:45.753849 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278fa27-86d3-440c-81d3-cd92e04b1a61 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:45.761386 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178b01e-e75e-4c94-9435-b739b43dc2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:45.762300 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178b01e-e75e-4c94-9435-b739b43dc2a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:45.762706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178b01e-e75e-4c94-9435-b739b43dc2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:45.763209 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178b01e-e75e-4c94-9435-b739b43dc2a1 admin admin] PATH_INFO: `/v3/projects/c9d0d90eccb5403eb3f93d7b1fec273c` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:45.764904 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178b01e-e75e-4c94-9435-b739b43dc2a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9d0d90eccb5403eb3f93d7b1fec273c)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:45.769583 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178b01e-e75e-4c94-9435-b739b43dc2a1 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:45.819442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:45.828595 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0178b01e-e75e-4c94-9435-b739b43dc2a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9d0d90eccb5403eb3f93d7b1fec273c', 'request_id': 'req-0178b01e-e75e-4c94-9435-b739b43dc2a1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:45.837248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0178b01e-e75e-4c94-9435-b739b43dc2a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9d0d90eccb5403eb3f93d7b1fec273c', 'request_id': 'req-0178b01e-e75e-4c94-9435-b739b43dc2a1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:45.846947 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0178b01e-e75e-4c94-9435-b739b43dc2a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9d0d90eccb5403eb3f93d7b1fec273c', 'request_id': 'req-0178b01e-e75e-4c94-9435-b739b43dc2a1'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:45.861619 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0178b01e-e75e-4c94-9435-b739b43dc2a1 admin admin] The token cache is being invalidate because project c9d0d90eccb5403eb3f93d7b1fec273c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:41:45.861919 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0178b01e-e75e-4c94-9435-b739b43dc2a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:45.862594 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-0178b01e-e75e-4c94-9435-b739b43dc2a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:45.864318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1310/2634] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:41:45 2022] DELETE /identity/v3/projects/c9d0d90eccb5403eb3f93d7b1fec273c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:45.868648 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32636c0d-6235-40f0-bd17-8aa000dfc80d service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:45.880738 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32636c0d-6235-40f0-bd17-8aa000dfc80d service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:45.953521 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c2fac-d7bc-417a-861f-a599729cac39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:45.955368 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c2fac-d7bc-417a-861f-a599729cac39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:45.956057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c2fac-d7bc-417a-861f-a599729cac39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:45.956395 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c2fac-d7bc-417a-861f-a599729cac39 admin admin] PATH_INFO: `/v3/users/5543f8f60fe0479796349a0c20977726` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:45.958489 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c2fac-d7bc-417a-861f-a599729cac39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5543f8f60fe0479796349a0c20977726)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:45.961227 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c2fac-d7bc-417a-861f-a599729cac39 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:46.022413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-e56c2fac-d7bc-417a-861f-a599729cac39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5543f8f60fe0479796349a0c20977726'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:46.030985 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-e56c2fac-d7bc-417a-861f-a599729cac39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5543f8f60fe0479796349a0c20977726'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:46.031876 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-e56c2fac-d7bc-417a-861f-a599729cac39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5543f8f60fe0479796349a0c20977726'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:46.040952 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-e56c2fac-d7bc-417a-861f-a599729cac39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5543f8f60fe0479796349a0c20977726'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:46.056918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-e56c2fac-d7bc-417a-861f-a599729cac39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5543f8f60fe0479796349a0c20977726'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:46.064404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1325/2635] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:41:45 2022] DELETE /identity/v3/users/5543f8f60fe0479796349a0c20977726 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:46.427146 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178b01e-e75e-4c94-9435-b739b43dc2a1 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:46.428796 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0178b01e-e75e-4c94-9435-b739b43dc2a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:46.447371 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178b01e-e75e-4c94-9435-b739b43dc2a1 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:46.528332 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc365e88-c65b-4e4b-ba0e-5e4f0015d1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:46.529257 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc365e88-c65b-4e4b-ba0e-5e4f0015d1b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:46.529553 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc365e88-c65b-4e4b-ba0e-5e4f0015d1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:46.529836 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc365e88-c65b-4e4b-ba0e-5e4f0015d1b9 admin admin] PATH_INFO: `/v3/projects/c898da3a09c64a2c85508992f08c9f35` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:46.531082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc365e88-c65b-4e4b-ba0e-5e4f0015d1b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c898da3a09c64a2c85508992f08c9f35)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:46.533862 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc365e88-c65b-4e4b-ba0e-5e4f0015d1b9 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:46.594683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-bc365e88-c65b-4e4b-ba0e-5e4f0015d1b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c898da3a09c64a2c85508992f08c9f35', 'request_id': 'req-bc365e88-c65b-4e4b-ba0e-5e4f0015d1b9'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:46.602847 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-bc365e88-c65b-4e4b-ba0e-5e4f0015d1b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c898da3a09c64a2c85508992f08c9f35', 'request_id': 'req-bc365e88-c65b-4e4b-ba0e-5e4f0015d1b9'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:46.609586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-bc365e88-c65b-4e4b-ba0e-5e4f0015d1b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c898da3a09c64a2c85508992f08c9f35', 'request_id': 'req-bc365e88-c65b-4e4b-ba0e-5e4f0015d1b9'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:46.621236 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-bc365e88-c65b-4e4b-ba0e-5e4f0015d1b9 admin admin] The token cache is being invalidate because project c898da3a09c64a2c85508992f08c9f35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:41:46.621532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-bc365e88-c65b-4e4b-ba0e-5e4f0015d1b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:46.622318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-bc365e88-c65b-4e4b-ba0e-5e4f0015d1b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:46.623944 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1311/2636] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:41:46 2022] DELETE /identity/v3/projects/c898da3a09c64a2c85508992f08c9f35 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:46.630125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad870e-a0df-49f9-aded-3ef784967882 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:46.630423 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad870e-a0df-49f9-aded-3ef784967882 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:46.630735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad870e-a0df-49f9-aded-3ef784967882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:46.653066 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-faad870e-a0df-49f9-aded-3ef784967882 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:46.654423 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-faad870e-a0df-49f9-aded-3ef784967882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:46.720921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1326/2637] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:46.728869 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e70cbb-9d9a-4ea3-a3ae-d79a551db233 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:46.729219 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e70cbb-9d9a-4ea3-a3ae-d79a551db233 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:46.729656 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e70cbb-9d9a-4ea3-a3ae-d79a551db233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:46.751102 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d1e70cbb-9d9a-4ea3-a3ae-d79a551db233 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:46.752669 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d1e70cbb-9d9a-4ea3-a3ae-d79a551db233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:46.762571 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1312/2638] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:46.767506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad870e-a0df-49f9-aded-3ef784967882 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:46.778809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad870e-a0df-49f9-aded-3ef784967882 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:46.785206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246ed5e-03ad-4059-b25b-0b5220eeaa1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:46.785888 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246ed5e-03ad-4059-b25b-0b5220eeaa1d admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:46.786187 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246ed5e-03ad-4059-b25b-0b5220eeaa1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:46.786465 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246ed5e-03ad-4059-b25b-0b5220eeaa1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:46.787260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246ed5e-03ad-4059-b25b-0b5220eeaa1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:41:46.787574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246ed5e-03ad-4059-b25b-0b5220eeaa1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:46.789191 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246ed5e-03ad-4059-b25b-0b5220eeaa1d admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:46.792029 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:46.805332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1327/2639] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:41:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:41:46.807907 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e70cbb-9d9a-4ea3-a3ae-d79a551db233 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:46.817604 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e70cbb-9d9a-4ea3-a3ae-d79a551db233 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:46.822183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a209d-32e5-45ca-aa33-1f812d80cfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:46.822837 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a209d-32e5-45ca-aa33-1f812d80cfba admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:46.823122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a209d-32e5-45ca-aa33-1f812d80cfba admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:46.823401 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a209d-32e5-45ca-aa33-1f812d80cfba admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:46.824102 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a209d-32e5-45ca-aa33-1f812d80cfba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:46.826306 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a209d-32e5-45ca-aa33-1f812d80cfba admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:46.855713 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1313/2640] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:41:46 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:46.860450 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246ed5e-03ad-4059-b25b-0b5220eeaa1d admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:46.870442 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246ed5e-03ad-4059-b25b-0b5220eeaa1d admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:46.929872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248878e-9490-4f05-9de6-945aba7854d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:46.930468 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248878e-9490-4f05-9de6-945aba7854d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:46.930763 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248878e-9490-4f05-9de6-945aba7854d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:46.931051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248878e-9490-4f05-9de6-945aba7854d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:46.931756 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f248878e-9490-4f05-9de6-945aba7854d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:46.933717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f248878e-9490-4f05-9de6-945aba7854d8 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:46.957971 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1328/2641] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:41:46 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:46.961936 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a209d-32e5-45ca-aa33-1f812d80cfba admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:46.972741 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a209d-32e5-45ca-aa33-1f812d80cfba admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:46.977543 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe16d4-510e-4d1d-bb12-8583150e1d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:46.978238 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe16d4-510e-4d1d-bb12-8583150e1d13 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:46.978509 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe16d4-510e-4d1d-bb12-8583150e1d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:46.978804 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe16d4-510e-4d1d-bb12-8583150e1d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:46.979521 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe16d4-510e-4d1d-bb12-8583150e1d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:46.981421 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe16d4-510e-4d1d-bb12-8583150e1d13 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:46.994007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1314/2642] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:46.998223 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248878e-9490-4f05-9de6-945aba7854d8 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:47.009082 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248878e-9490-4f05-9de6-945aba7854d8 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:47.015475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86152626-16a6-4a16-b975-46b4be4b0acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:47.016796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86152626-16a6-4a16-b975-46b4be4b0acd admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:47.017258 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86152626-16a6-4a16-b975-46b4be4b0acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:47.017548 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86152626-16a6-4a16-b975-46b4be4b0acd admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:47.018280 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86152626-16a6-4a16-b975-46b4be4b0acd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:47.019875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86152626-16a6-4a16-b975-46b4be4b0acd admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:47.029938 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1329/2643] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:41:47.034423 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe16d4-510e-4d1d-bb12-8583150e1d13 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:47.044963 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe16d4-510e-4d1d-bb12-8583150e1d13 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:47.050199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2ca75-cf11-4d95-9d15-0576f32c4c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:47.051089 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2ca75-cf11-4d95-9d15-0576f32c4c1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:47.051383 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2ca75-cf11-4d95-9d15-0576f32c4c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:47.051668 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2ca75-cf11-4d95-9d15-0576f32c4c1d admin admin] PATH_INFO: `/v3/projects/345a270d4bab4e1da43d1f6add2de914/users/0587b2764da045eb85d06570bcd25609/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:47.067538 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2ca75-cf11-4d95-9d15-0576f32c4c1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=345a270d4bab4e1da43d1f6add2de914, user_id=0587b2764da045eb85d06570bcd25609, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:47.070267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2ca75-cf11-4d95-9d15-0576f32c4c1d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:47.072549 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:47.085352 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1315/2644] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:47 2022] PUT /identity/v3/projects/345a270d4bab4e1da43d1f6add2de914/users/0587b2764da045eb85d06570bcd25609/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:47.096881 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a338c-ac7a-432d-805b-5252b3306e18 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:47.097505 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a338c-ac7a-432d-805b-5252b3306e18 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:47.097633 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a338c-ac7a-432d-805b-5252b3306e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:47.126813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-6f3a338c-ac7a-432d-805b-5252b3306e18 None None] MFA Rules not processed for user `0587b2764da045eb85d06570bcd25609`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:47.128047 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-6f3a338c-ac7a-432d-805b-5252b3306e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:47.196135 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1330/2645] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:47.200647 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2ca75-cf11-4d95-9d15-0576f32c4c1d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:47.265990 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2ca75-cf11-4d95-9d15-0576f32c4c1d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:47.272635 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105c51c-88cd-4eb9-a9d9-7827980a914b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:47.273342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105c51c-88cd-4eb9-a9d9-7827980a914b admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:47.273627 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105c51c-88cd-4eb9-a9d9-7827980a914b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:47.273904 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105c51c-88cd-4eb9-a9d9-7827980a914b admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:47.274605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c105c51c-88cd-4eb9-a9d9-7827980a914b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:47.276519 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c105c51c-88cd-4eb9-a9d9-7827980a914b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:47.303045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1316/2646] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:41:47 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:47.307585 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a338c-ac7a-432d-805b-5252b3306e18 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:47.319459 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a338c-ac7a-432d-805b-5252b3306e18 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:47.378748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1086412-9d3e-4d89-a313-3d1274a0fba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:47.379434 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1086412-9d3e-4d89-a313-3d1274a0fba0 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:47.379729 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1086412-9d3e-4d89-a313-3d1274a0fba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:47.380032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1086412-9d3e-4d89-a313-3d1274a0fba0 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:47.380732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1086412-9d3e-4d89-a313-3d1274a0fba0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:47.382704 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1086412-9d3e-4d89-a313-3d1274a0fba0 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:47.391444 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:47.405721 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1331/2647] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:41:47 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:47.410304 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105c51c-88cd-4eb9-a9d9-7827980a914b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:47.421687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105c51c-88cd-4eb9-a9d9-7827980a914b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:47.427531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a569de-b391-459a-8be5-a90b251ff905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:47.428212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a569de-b391-459a-8be5-a90b251ff905 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:47.428502 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a569de-b391-459a-8be5-a90b251ff905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:47.428782 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a569de-b391-459a-8be5-a90b251ff905 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:47.429556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a569de-b391-459a-8be5-a90b251ff905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:47.431155 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a569de-b391-459a-8be5-a90b251ff905 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:47.442774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1317/2648] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:47.447028 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1086412-9d3e-4d89-a313-3d1274a0fba0 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:47.457577 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1086412-9d3e-4d89-a313-3d1274a0fba0 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:47.462342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738e4e5-c4d0-4fc0-ae32-3b1a71b29843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:47.463292 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738e4e5-c4d0-4fc0-ae32-3b1a71b29843 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:47.463584 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738e4e5-c4d0-4fc0-ae32-3b1a71b29843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:47.463869 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738e4e5-c4d0-4fc0-ae32-3b1a71b29843 admin admin] PATH_INFO: `/v3/projects/d174324f83544e9da9a63607b8eda634/users/3ffd0887f0e64810a7f3b9e4b9e68c5c/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:47.480844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738e4e5-c4d0-4fc0-ae32-3b1a71b29843 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d174324f83544e9da9a63607b8eda634, user_id=3ffd0887f0e64810a7f3b9e4b9e68c5c, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:47.483850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738e4e5-c4d0-4fc0-ae32-3b1a71b29843 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:47.499876 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1332/2649] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:47 2022] PUT /identity/v3/projects/d174324f83544e9da9a63607b8eda634/users/3ffd0887f0e64810a7f3b9e4b9e68c5c/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:47.504652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a569de-b391-459a-8be5-a90b251ff905 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:47.515290 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a569de-b391-459a-8be5-a90b251ff905 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:47.578630 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f9ea94-f63c-4cd1-aebc-45141e7245ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:47.579983 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9ea94-f63c-4cd1-aebc-45141e7245ca admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:47.580296 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9ea94-f63c-4cd1-aebc-45141e7245ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:47.580580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9ea94-f63c-4cd1-aebc-45141e7245ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:47.581366 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f9ea94-f63c-4cd1-aebc-45141e7245ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:47.583008 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f9ea94-f63c-4cd1-aebc-45141e7245ca admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:47.593940 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1318/2650] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:47.598456 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738e4e5-c4d0-4fc0-ae32-3b1a71b29843 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:47.609588 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738e4e5-c4d0-4fc0-ae32-3b1a71b29843 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:47.615724 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241da5b-cdf8-41c4-9dc5-0cc33e7ce75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:47.616755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241da5b-cdf8-41c4-9dc5-0cc33e7ce75a admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:47.617051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241da5b-cdf8-41c4-9dc5-0cc33e7ce75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:47.617374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241da5b-cdf8-41c4-9dc5-0cc33e7ce75a admin admin] PATH_INFO: `/v3/projects/d174324f83544e9da9a63607b8eda634/users/3ffd0887f0e64810a7f3b9e4b9e68c5c/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:47.619458 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241da5b-cdf8-41c4-9dc5-0cc33e7ce75a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d174324f83544e9da9a63607b8eda634, user_id=3ffd0887f0e64810a7f3b9e4b9e68c5c, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:47.622348 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241da5b-cdf8-41c4-9dc5-0cc33e7ce75a admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:47.628266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f9ea94-f63c-4cd1-aebc-45141e7245ca admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:47.630051 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-20f9ea94-f63c-4cd1-aebc-45141e7245ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:47.635416 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1333/2651] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:47 2022] PUT /identity/v3/projects/d174324f83544e9da9a63607b8eda634/users/3ffd0887f0e64810a7f3b9e4b9e68c5c/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:41:47.644594 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f9ea94-f63c-4cd1-aebc-45141e7245ca admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:47.646833 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca7d15-b5af-404f-92dd-486d00aaae72 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:47.647128 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca7d15-b5af-404f-92dd-486d00aaae72 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:47.647409 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca7d15-b5af-404f-92dd-486d00aaae72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:47.678571 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-d4ca7d15-b5af-404f-92dd-486d00aaae72 None None] MFA Rules not processed for user `3ffd0887f0e64810a7f3b9e4b9e68c5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:47.679868 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-d4ca7d15-b5af-404f-92dd-486d00aaae72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:47.717583 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de1033-f5ad-4528-afdf-f93dfba7362d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:47.719753 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de1033-f5ad-4528-afdf-f93dfba7362d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:47.720891 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de1033-f5ad-4528-afdf-f93dfba7362d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:47.721513 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de1033-f5ad-4528-afdf-f93dfba7362d admin admin] PATH_INFO: `/v3/users/4d99ef97227f40298df77e8c87e4add4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:47.723564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de1033-f5ad-4528-afdf-f93dfba7362d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d99ef97227f40298df77e8c87e4add4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:47.725555 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de1033-f5ad-4528-afdf-f93dfba7362d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:47.731611 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:47.771970 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1334/2652] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:41:47.782632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-57de1033-f5ad-4528-afdf-f93dfba7362d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d99ef97227f40298df77e8c87e4add4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:47.783660 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca7d15-b5af-404f-92dd-486d00aaae72 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:47.785081 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-d4ca7d15-b5af-404f-92dd-486d00aaae72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:47.795167 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-57de1033-f5ad-4528-afdf-f93dfba7362d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d99ef97227f40298df77e8c87e4add4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:47.795944 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-57de1033-f5ad-4528-afdf-f93dfba7362d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d99ef97227f40298df77e8c87e4add4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:47.805515 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-57de1033-f5ad-4528-afdf-f93dfba7362d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d99ef97227f40298df77e8c87e4add4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:47.816109 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-57de1033-f5ad-4528-afdf-f93dfba7362d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d99ef97227f40298df77e8c87e4add4'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:47.823066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1319/2653] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:41:47 2022] DELETE /identity/v3/users/4d99ef97227f40298df77e8c87e4add4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:47.827465 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ffd05-2552-4aa6-8875-14261de0671b None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:47.827882 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ffd05-2552-4aa6-8875-14261de0671b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:47.828290 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ffd05-2552-4aa6-8875-14261de0671b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:47.848418 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-651ffd05-2552-4aa6-8875-14261de0671b None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:47.849841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-651ffd05-2552-4aa6-8875-14261de0671b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:47.855281 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca7d15-b5af-404f-92dd-486d00aaae72 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:47.906438 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67178d42-a212-4a9e-96c4-7de9eded0166 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:47.907669 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67178d42-a212-4a9e-96c4-7de9eded0166 service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:47.907669 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67178d42-a212-4a9e-96c4-7de9eded0166 service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:47.908011 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67178d42-a212-4a9e-96c4-7de9eded0166 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:47.909859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-67178d42-a212-4a9e-96c4-7de9eded0166 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:47.911599 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1320/2654] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:41:47.921382 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ffd05-2552-4aa6-8875-14261de0671b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:47.921596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:47.923364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-651ffd05-2552-4aa6-8875-14261de0671b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:47.979831 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67178d42-a212-4a9e-96c4-7de9eded0166 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:47.981682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67178d42-a212-4a9e-96c4-7de9eded0166 service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:47.984683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ffd05-2552-4aa6-8875-14261de0671b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:47.989883 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a07d0-efbe-4a85-9a95-39bd697b6a97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:47.990698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a07d0-efbe-4a85-9a95-39bd697b6a97 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:47.991069 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a07d0-efbe-4a85-9a95-39bd697b6a97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:47.991373 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a07d0-efbe-4a85-9a95-39bd697b6a97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:47.993084 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1335/2655] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:41:48.002833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a07d0-efbe-4a85-9a95-39bd697b6a97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:48.005066 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a07d0-efbe-4a85-9a95-39bd697b6a97 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:48.017370 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1321/2656] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:41:48.351928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67178d42-a212-4a9e-96c4-7de9eded0166 service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:48.353618 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-67178d42-a212-4a9e-96c4-7de9eded0166 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:48.369480 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67178d42-a212-4a9e-96c4-7de9eded0166 service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:48.375673 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f824112-e42f-4969-aedf-d81b0f284296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:48.377340 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f824112-e42f-4969-aedf-d81b0f284296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:48.377657 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f824112-e42f-4969-aedf-d81b0f284296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:48.378042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f824112-e42f-4969-aedf-d81b0f284296 admin admin] PATH_INFO: `/v3/projects/83a70bfe861e4ecfae1ddcff044e0582` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:48.379314 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f824112-e42f-4969-aedf-d81b0f284296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83a70bfe861e4ecfae1ddcff044e0582)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:48.381414 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f824112-e42f-4969-aedf-d81b0f284296 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:48.439621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-8f824112-e42f-4969-aedf-d81b0f284296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83a70bfe861e4ecfae1ddcff044e0582', 'request_id': 'req-8f824112-e42f-4969-aedf-d81b0f284296'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:48.445918 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-8f824112-e42f-4969-aedf-d81b0f284296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83a70bfe861e4ecfae1ddcff044e0582', 'request_id': 'req-8f824112-e42f-4969-aedf-d81b0f284296'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:48.453063 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-8f824112-e42f-4969-aedf-d81b0f284296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83a70bfe861e4ecfae1ddcff044e0582', 'request_id': 'req-8f824112-e42f-4969-aedf-d81b0f284296'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:48.460232 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a07d0-efbe-4a85-9a95-39bd697b6a97 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:48.462045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1d7a07d0-efbe-4a85-9a95-39bd697b6a97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:48.468824 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-8f824112-e42f-4969-aedf-d81b0f284296 admin admin] The token cache is being invalidate because project 83a70bfe861e4ecfae1ddcff044e0582 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:41:48.469141 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-8f824112-e42f-4969-aedf-d81b0f284296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:48.469796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-8f824112-e42f-4969-aedf-d81b0f284296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:48.471604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1336/2657] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:41:48 2022] DELETE /identity/v3/projects/83a70bfe861e4ecfae1ddcff044e0582 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:48.476270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b04466-dd88-451b-8499-91b6a6bec731 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:48.476563 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b04466-dd88-451b-8499-91b6a6bec731 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:48.476679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a07d0-efbe-4a85-9a95-39bd697b6a97 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:48.476847 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b04466-dd88-451b-8499-91b6a6bec731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:48.496104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-83b04466-dd88-451b-8499-91b6a6bec731 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:48.497415 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-83b04466-dd88-451b-8499-91b6a6bec731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:48.504185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:48.541133 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27bdf5-0e72-4427-b759-cd12f2abae27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:48.542054 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27bdf5-0e72-4427-b759-cd12f2abae27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:48.542392 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27bdf5-0e72-4427-b759-cd12f2abae27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:48.542629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27bdf5-0e72-4427-b759-cd12f2abae27 admin admin] PATH_INFO: `/v3/users/0587b2764da045eb85d06570bcd25609` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:48.544624 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b27bdf5-0e72-4427-b759-cd12f2abae27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0587b2764da045eb85d06570bcd25609)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:48.546700 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b27bdf5-0e72-4427-b759-cd12f2abae27 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:48.559822 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1337/2658] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:41:48.568272 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19126b-dfe1-4f59-9153-3cb2667ce162 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:48.568581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19126b-dfe1-4f59-9153-3cb2667ce162 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:48.568854 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19126b-dfe1-4f59-9153-3cb2667ce162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:48.594950 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-df19126b-dfe1-4f59-9153-3cb2667ce162 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:48.596181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-df19126b-dfe1-4f59-9153-3cb2667ce162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:48.606337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1338/2659] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:48.606826 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4b27bdf5-0e72-4427-b759-cd12f2abae27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0587b2764da045eb85d06570bcd25609'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:48.614764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19126b-dfe1-4f59-9153-3cb2667ce162 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:48.624759 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19126b-dfe1-4f59-9153-3cb2667ce162 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:48.627571 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4b27bdf5-0e72-4427-b759-cd12f2abae27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0587b2764da045eb85d06570bcd25609'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:48.628296 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4b27bdf5-0e72-4427-b759-cd12f2abae27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0587b2764da045eb85d06570bcd25609'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:48.631396 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165c88f-e6cd-4396-a86f-5c133c581e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:48.632077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165c88f-e6cd-4396-a86f-5c133c581e9f admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:48.632374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165c88f-e6cd-4396-a86f-5c133c581e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:48.632655 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165c88f-e6cd-4396-a86f-5c133c581e9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:48.633453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165c88f-e6cd-4396-a86f-5c133c581e9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:41:48.633772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165c88f-e6cd-4396-a86f-5c133c581e9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:48.635445 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165c88f-e6cd-4396-a86f-5c133c581e9f admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:48.636051 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4b27bdf5-0e72-4427-b759-cd12f2abae27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0587b2764da045eb85d06570bcd25609'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:48.645516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4b27bdf5-0e72-4427-b759-cd12f2abae27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0587b2764da045eb85d06570bcd25609'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:48.649032 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1339/2660] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:41:48.652874 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1322/2661] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:41:48 2022] DELETE /identity/v3/users/0587b2764da045eb85d06570bcd25609 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:41:48.656240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27bdf5-0e72-4427-b759-cd12f2abae27 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:48.657026 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8a753-468d-43f1-bd12-116e410faabe None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:48.657343 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8a753-468d-43f1-bd12-116e410faabe None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:48.657649 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8a753-468d-43f1-bd12-116e410faabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:48.657920 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4b27bdf5-0e72-4427-b759-cd12f2abae27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:48.677266 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-5cc8a753-468d-43f1-bd12-116e410faabe None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:48.678513 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-5cc8a753-468d-43f1-bd12-116e410faabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:48.684026 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:48.730034 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27bdf5-0e72-4427-b759-cd12f2abae27 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:48.736184 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b322e2-eb16-45ef-9654-9d770481370e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:48.736901 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b322e2-eb16-45ef-9654-9d770481370e admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:48.737219 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b322e2-eb16-45ef-9654-9d770481370e admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:48.737507 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b322e2-eb16-45ef-9654-9d770481370e admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:48.738222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b322e2-eb16-45ef-9654-9d770481370e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:48.740156 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b322e2-eb16-45ef-9654-9d770481370e admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:48.744983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1340/2662] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:48.757025 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8a753-468d-43f1-bd12-116e410faabe None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:48.758490 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-5cc8a753-468d-43f1-bd12-116e410faabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:48.770738 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1323/2663] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:41:48 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:48.774763 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b322e2-eb16-45ef-9654-9d770481370e admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:48.818960 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:48.837782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8a753-468d-43f1-bd12-116e410faabe None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:48.848685 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b322e2-eb16-45ef-9654-9d770481370e admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:48.853356 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6409c-30b5-4634-993f-b8143ad064f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:48.854031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6409c-30b5-4634-993f-b8143ad064f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:48.854317 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6409c-30b5-4634-993f-b8143ad064f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:48.854599 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6409c-30b5-4634-993f-b8143ad064f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:48.855293 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6409c-30b5-4634-993f-b8143ad064f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:48.857620 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6409c-30b5-4634-993f-b8143ad064f7 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:48.883016 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1324/2664] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:41:48 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:48.884081 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6409c-30b5-4634-993f-b8143ad064f7 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:48.895485 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6409c-30b5-4634-993f-b8143ad064f7 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:48.900301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf379e38-d2df-4f68-8acd-66acd60bbb4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:48.900981 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf379e38-d2df-4f68-8acd-66acd60bbb4a service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:48.901308 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf379e38-d2df-4f68-8acd-66acd60bbb4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:48.901745 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf379e38-d2df-4f68-8acd-66acd60bbb4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:48.901913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01373e-d520-437f-ac7f-7391d0d35428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:48.902601 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01373e-d520-437f-ac7f-7391d0d35428 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:48.902907 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01373e-d520-437f-ac7f-7391d0d35428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:48.903186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01373e-d520-437f-ac7f-7391d0d35428 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:48.904015 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db01373e-d520-437f-ac7f-7391d0d35428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:48.905607 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db01373e-d520-437f-ac7f-7391d0d35428 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:48.916657 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf379e38-d2df-4f68-8acd-66acd60bbb4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:48.918461 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1325/2665] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:48.921434 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf379e38-d2df-4f68-8acd-66acd60bbb4a service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:48.923899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01373e-d520-437f-ac7f-7391d0d35428 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:48.934416 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01373e-d520-437f-ac7f-7391d0d35428 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:48.935764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1341/2666] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:41:48.942403 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab1f84-9dc9-420e-a1ce-c827fde6e23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:48.943181 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab1f84-9dc9-420e-a1ce-c827fde6e23c admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:48.943541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab1f84-9dc9-420e-a1ce-c827fde6e23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:48.943896 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab1f84-9dc9-420e-a1ce-c827fde6e23c admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:48.944681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab1f84-9dc9-420e-a1ce-c827fde6e23c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:48.946347 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab1f84-9dc9-420e-a1ce-c827fde6e23c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:48.958026 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1326/2667] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:41:48.962386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf379e38-d2df-4f68-8acd-66acd60bbb4a service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:48.973644 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf379e38-d2df-4f68-8acd-66acd60bbb4a service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:48.979095 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6847e757-6e68-4886-a596-1ec20e8fbca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:48.980051 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847e757-6e68-4886-a596-1ec20e8fbca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:48.980349 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847e757-6e68-4886-a596-1ec20e8fbca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:48.980636 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847e757-6e68-4886-a596-1ec20e8fbca1 admin admin] PATH_INFO: `/v3/projects/706c5aa484c8430ca38159cca5d591d2/users/1233ca8155fe4be3a6d805299e5143fd/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:49.001697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6847e757-6e68-4886-a596-1ec20e8fbca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=706c5aa484c8430ca38159cca5d591d2, user_id=1233ca8155fe4be3a6d805299e5143fd, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:49.004516 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6847e757-6e68-4886-a596-1ec20e8fbca1 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:49.019616 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1342/2668] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:48 2022] PUT /identity/v3/projects/706c5aa484c8430ca38159cca5d591d2/users/1233ca8155fe4be3a6d805299e5143fd/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:49.030777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738912d-146a-436d-ad5e-10ec99eecea4 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:49.031078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738912d-146a-436d-ad5e-10ec99eecea4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:49.031367 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738912d-146a-436d-ad5e-10ec99eecea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:49.060984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-5738912d-146a-436d-ad5e-10ec99eecea4 None None] MFA Rules not processed for user `1233ca8155fe4be3a6d805299e5143fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:49.062320 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5738912d-146a-436d-ad5e-10ec99eecea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:49.139964 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1327/2669] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:41:49.151999 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6847e757-6e68-4886-a596-1ec20e8fbca1 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:49.154494 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-6847e757-6e68-4886-a596-1ec20e8fbca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:49.213884 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:49.221840 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6847e757-6e68-4886-a596-1ec20e8fbca1 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:49.225870 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738912d-146a-436d-ad5e-10ec99eecea4 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:49.227077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce10db-f452-4f8b-b136-e041fe019f11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:49.227520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-5738912d-146a-436d-ad5e-10ec99eecea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:49.227879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce10db-f452-4f8b-b136-e041fe019f11 service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:49.228225 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce10db-f452-4f8b-b136-e041fe019f11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:49.228513 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce10db-f452-4f8b-b136-e041fe019f11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:49.250948 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce10db-f452-4f8b-b136-e041fe019f11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:49.252546 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce10db-f452-4f8b-b136-e041fe019f11 service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:49.265068 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1343/2670] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:49.338470 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738912d-146a-436d-ad5e-10ec99eecea4 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:49.343546 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad124ebe-806e-4ec4-8545-9c2b8330196d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:49.344952 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad124ebe-806e-4ec4-8545-9c2b8330196d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:49.345354 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad124ebe-806e-4ec4-8545-9c2b8330196d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:49.345642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad124ebe-806e-4ec4-8545-9c2b8330196d admin admin] PATH_INFO: `/v3/projects/345a270d4bab4e1da43d1f6add2de914` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:49.346871 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad124ebe-806e-4ec4-8545-9c2b8330196d admin admin] RBAC: Authorizing `identity:delete_project(project_id=345a270d4bab4e1da43d1f6add2de914)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:49.348809 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad124ebe-806e-4ec4-8545-9c2b8330196d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:49.389230 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:49.408485 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ad124ebe-806e-4ec4-8545-9c2b8330196d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '345a270d4bab4e1da43d1f6add2de914', 'request_id': 'req-ad124ebe-806e-4ec4-8545-9c2b8330196d'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:49.416779 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ad124ebe-806e-4ec4-8545-9c2b8330196d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '345a270d4bab4e1da43d1f6add2de914', 'request_id': 'req-ad124ebe-806e-4ec4-8545-9c2b8330196d'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:49.425120 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ad124ebe-806e-4ec4-8545-9c2b8330196d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '345a270d4bab4e1da43d1f6add2de914', 'request_id': 'req-ad124ebe-806e-4ec4-8545-9c2b8330196d'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:49.438724 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ad124ebe-806e-4ec4-8545-9c2b8330196d admin admin] The token cache is being invalidate because project 345a270d4bab4e1da43d1f6add2de914 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:41:49.439643 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ad124ebe-806e-4ec4-8545-9c2b8330196d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:49.440615 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-ad124ebe-806e-4ec4-8545-9c2b8330196d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:49.443300 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1328/2671] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:41:49 2022] DELETE /identity/v3/projects/345a270d4bab4e1da43d1f6add2de914 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:49.448003 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce10db-f452-4f8b-b136-e041fe019f11 service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:49.459922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce10db-f452-4f8b-b136-e041fe019f11 service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:49.529933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720321a-1805-4497-a65b-c61064b42f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:49.530949 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720321a-1805-4497-a65b-c61064b42f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:49.531223 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720321a-1805-4497-a65b-c61064b42f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:49.532201 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720321a-1805-4497-a65b-c61064b42f33 admin admin] PATH_INFO: `/v3/users/3ffd0887f0e64810a7f3b9e4b9e68c5c` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:49.533611 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720321a-1805-4497-a65b-c61064b42f33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ffd0887f0e64810a7f3b9e4b9e68c5c)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:49.538145 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720321a-1805-4497-a65b-c61064b42f33 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:49.604003 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d720321a-1805-4497-a65b-c61064b42f33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ffd0887f0e64810a7f3b9e4b9e68c5c'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:49.612054 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d720321a-1805-4497-a65b-c61064b42f33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ffd0887f0e64810a7f3b9e4b9e68c5c'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:49.612876 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d720321a-1805-4497-a65b-c61064b42f33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ffd0887f0e64810a7f3b9e4b9e68c5c'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:49.620369 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d720321a-1805-4497-a65b-c61064b42f33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ffd0887f0e64810a7f3b9e4b9e68c5c'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:49.634778 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d720321a-1805-4497-a65b-c61064b42f33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ffd0887f0e64810a7f3b9e4b9e68c5c'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:49.642070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1344/2672] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:41:49 2022] DELETE /identity/v3/users/3ffd0887f0e64810a7f3b9e4b9e68c5c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:49.934435 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad124ebe-806e-4ec4-8545-9c2b8330196d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:49.936016 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-ad124ebe-806e-4ec4-8545-9c2b8330196d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:49.952148 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad124ebe-806e-4ec4-8545-9c2b8330196d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:50.015412 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068aa04-edb9-494a-a246-c4e8cb0b479c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:50.016925 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068aa04-edb9-494a-a246-c4e8cb0b479c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:50.017282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068aa04-edb9-494a-a246-c4e8cb0b479c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:50.017560 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068aa04-edb9-494a-a246-c4e8cb0b479c admin admin] PATH_INFO: `/v3/projects/d174324f83544e9da9a63607b8eda634` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:50.018813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068aa04-edb9-494a-a246-c4e8cb0b479c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d174324f83544e9da9a63607b8eda634)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:50.020730 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068aa04-edb9-494a-a246-c4e8cb0b479c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:50.071011 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e068aa04-edb9-494a-a246-c4e8cb0b479c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd174324f83544e9da9a63607b8eda634', 'request_id': 'req-e068aa04-edb9-494a-a246-c4e8cb0b479c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:50.079228 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e068aa04-edb9-494a-a246-c4e8cb0b479c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd174324f83544e9da9a63607b8eda634', 'request_id': 'req-e068aa04-edb9-494a-a246-c4e8cb0b479c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:50.087267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e068aa04-edb9-494a-a246-c4e8cb0b479c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd174324f83544e9da9a63607b8eda634', 'request_id': 'req-e068aa04-edb9-494a-a246-c4e8cb0b479c'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:50.098456 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e068aa04-edb9-494a-a246-c4e8cb0b479c admin admin] The token cache is being invalidate because project d174324f83544e9da9a63607b8eda634 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:41:50.098753 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e068aa04-edb9-494a-a246-c4e8cb0b479c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:50.099418 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e068aa04-edb9-494a-a246-c4e8cb0b479c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:50.100968 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1329/2673] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:41:49 2022] DELETE /identity/v3/projects/d174324f83544e9da9a63607b8eda634 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:50.106133 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f3a20-e15a-4683-8371-13ea344a56e1 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:50.106439 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f3a20-e15a-4683-8371-13ea344a56e1 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:50.106729 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f3a20-e15a-4683-8371-13ea344a56e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:50.124717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-586f3a20-e15a-4683-8371-13ea344a56e1 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:50.126700 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-586f3a20-e15a-4683-8371-13ea344a56e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:50.194506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1345/2674] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:50.203500 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43af70-f2d0-48c0-a7b3-ace4626fe761 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:50.203938 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43af70-f2d0-48c0-a7b3-ace4626fe761 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:50.204363 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43af70-f2d0-48c0-a7b3-ace4626fe761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:50.224637 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-9e43af70-f2d0-48c0-a7b3-ace4626fe761 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:50.225951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9e43af70-f2d0-48c0-a7b3-ace4626fe761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:50.234572 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1330/2675] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:50.239090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f3a20-e15a-4683-8371-13ea344a56e1 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:50.248904 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f3a20-e15a-4683-8371-13ea344a56e1 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:50.254316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca6f18-e25e-485c-b1b8-ccdaccc6bda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:50.255095 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca6f18-e25e-485c-b1b8-ccdaccc6bda0 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:50.255394 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca6f18-e25e-485c-b1b8-ccdaccc6bda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:50.255677 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca6f18-e25e-485c-b1b8-ccdaccc6bda0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:50.256427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca6f18-e25e-485c-b1b8-ccdaccc6bda0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:41:50.256743 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca6f18-e25e-485c-b1b8-ccdaccc6bda0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:50.258393 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca6f18-e25e-485c-b1b8-ccdaccc6bda0 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:50.260996 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:50.272749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1346/2676] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:41:50.276972 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43af70-f2d0-48c0-a7b3-ace4626fe761 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:50.286812 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43af70-f2d0-48c0-a7b3-ace4626fe761 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:50.291482 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1eda1-30a1-49d8-90a2-393ec9b2562d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:50.292177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1eda1-30a1-49d8-90a2-393ec9b2562d admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:50.292467 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1eda1-30a1-49d8-90a2-393ec9b2562d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:50.292746 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1eda1-30a1-49d8-90a2-393ec9b2562d admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:50.293547 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d1eda1-30a1-49d8-90a2-393ec9b2562d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:50.295464 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d1eda1-30a1-49d8-90a2-393ec9b2562d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:50.322790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1331/2677] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:41:50 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:50.326289 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca6f18-e25e-485c-b1b8-ccdaccc6bda0 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:50.336503 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca6f18-e25e-485c-b1b8-ccdaccc6bda0 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:50.395945 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea508db0-5786-4892-a47d-2ab03e6cac46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:50.396632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea508db0-5786-4892-a47d-2ab03e6cac46 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:50.396922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea508db0-5786-4892-a47d-2ab03e6cac46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:50.397233 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea508db0-5786-4892-a47d-2ab03e6cac46 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:50.397947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea508db0-5786-4892-a47d-2ab03e6cac46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:50.399898 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea508db0-5786-4892-a47d-2ab03e6cac46 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:50.421277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1347/2678] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:41:50 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:50.426014 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1eda1-30a1-49d8-90a2-393ec9b2562d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:50.437263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1eda1-30a1-49d8-90a2-393ec9b2562d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:50.443095 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315252ca-118c-45bd-bfbe-c4e34e4d0d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:50.443773 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315252ca-118c-45bd-bfbe-c4e34e4d0d8b admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:50.444061 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315252ca-118c-45bd-bfbe-c4e34e4d0d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:50.444343 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315252ca-118c-45bd-bfbe-c4e34e4d0d8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:50.445078 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315252ca-118c-45bd-bfbe-c4e34e4d0d8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:50.446725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315252ca-118c-45bd-bfbe-c4e34e4d0d8b admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:50.458291 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1332/2679] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:50.462487 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea508db0-5786-4892-a47d-2ab03e6cac46 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:50.473358 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea508db0-5786-4892-a47d-2ab03e6cac46 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:50.478007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa5151-1f32-4f24-ba07-2ef42f398dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:50.478683 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa5151-1f32-4f24-ba07-2ef42f398dfd admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:50.478972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa5151-1f32-4f24-ba07-2ef42f398dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:50.479252 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa5151-1f32-4f24-ba07-2ef42f398dfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:50.480014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa5151-1f32-4f24-ba07-2ef42f398dfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:50.481945 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa5151-1f32-4f24-ba07-2ef42f398dfd admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:50.492839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1348/2680] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:50.497135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315252ca-118c-45bd-bfbe-c4e34e4d0d8b admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:50.512520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315252ca-118c-45bd-bfbe-c4e34e4d0d8b admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:50.518980 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f4e92-2e6f-474a-aaa7-b2d5afeb6a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:50.520152 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f4e92-2e6f-474a-aaa7-b2d5afeb6a68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:50.520909 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f4e92-2e6f-474a-aaa7-b2d5afeb6a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:50.520909 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f4e92-2e6f-474a-aaa7-b2d5afeb6a68 admin admin] PATH_INFO: `/v3/projects/37df6c266bac4dce818dc346255b53d2/users/e10b8d7752834700b81dc24fc3445e0b/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:50.525038 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:50.543487 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f4e92-2e6f-474a-aaa7-b2d5afeb6a68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37df6c266bac4dce818dc346255b53d2, user_id=e10b8d7752834700b81dc24fc3445e0b, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:50.547284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f4e92-2e6f-474a-aaa7-b2d5afeb6a68 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:50.564742 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1333/2681] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:50 2022] PUT /identity/v3/projects/37df6c266bac4dce818dc346255b53d2/users/e10b8d7752834700b81dc24fc3445e0b/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:50.576186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2a410-c28f-4bc1-b99c-251156e4c2bb None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:50.576574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2a410-c28f-4bc1-b99c-251156e4c2bb None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:50.576984 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2a410-c28f-4bc1-b99c-251156e4c2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:50.614310 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-87d2a410-c28f-4bc1-b99c-251156e4c2bb None None] MFA Rules not processed for user `e10b8d7752834700b81dc24fc3445e0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:50.616194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-87d2a410-c28f-4bc1-b99c-251156e4c2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:50.715252 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1349/2682] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:50.727384 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f4e92-2e6f-474a-aaa7-b2d5afeb6a68 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:50.729381 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d72f4e92-2e6f-474a-aaa7-b2d5afeb6a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:50.797812 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f4e92-2e6f-474a-aaa7-b2d5afeb6a68 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:50.801839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1509f0-88aa-45aa-8b6f-806da5d7012c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:50.802624 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1509f0-88aa-45aa-8b6f-806da5d7012c service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:50.802805 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1509f0-88aa-45aa-8b6f-806da5d7012c service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:50.803087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1509f0-88aa-45aa-8b6f-806da5d7012c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:50.812522 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1509f0-88aa-45aa-8b6f-806da5d7012c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:50.814895 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1509f0-88aa-45aa-8b6f-806da5d7012c service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:50.825335 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1334/2683] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:41:50.846983 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2a410-c28f-4bc1-b99c-251156e4c2bb None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:50.947909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2a410-c28f-4bc1-b99c-251156e4c2bb None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:50.953722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b1e92-229a-415d-afc2-0044fab76baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:50.955016 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b1e92-229a-415d-afc2-0044fab76baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:50.955308 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b1e92-229a-415d-afc2-0044fab76baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:50.955590 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b1e92-229a-415d-afc2-0044fab76baf admin admin] PATH_INFO: `/v3/users/e10b8d7752834700b81dc24fc3445e0b` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:50.957199 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b1e92-229a-415d-afc2-0044fab76baf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e10b8d7752834700b81dc24fc3445e0b)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:50.959138 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b1e92-229a-415d-afc2-0044fab76baf admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:50.964626 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:51.030916 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-5f1b1e92-229a-415d-afc2-0044fab76baf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e10b8d7752834700b81dc24fc3445e0b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:51.041808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-5f1b1e92-229a-415d-afc2-0044fab76baf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e10b8d7752834700b81dc24fc3445e0b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:51.042459 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-5f1b1e92-229a-415d-afc2-0044fab76baf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e10b8d7752834700b81dc24fc3445e0b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:51.059297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-5f1b1e92-229a-415d-afc2-0044fab76baf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e10b8d7752834700b81dc24fc3445e0b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:51.075398 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-5f1b1e92-229a-415d-afc2-0044fab76baf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e10b8d7752834700b81dc24fc3445e0b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:51.082230 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1350/2684] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:41:50 2022] DELETE /identity/v3/users/e10b8d7752834700b81dc24fc3445e0b => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:51.086926 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4fb47-bb43-4c32-83a5-185a383fb1fa None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:51.087422 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4fb47-bb43-4c32-83a5-185a383fb1fa None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:51.087422 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4fb47-bb43-4c32-83a5-185a383fb1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:51.105368 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-b4d4fb47-bb43-4c32-83a5-185a383fb1fa None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:51.107314 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b4d4fb47-bb43-4c32-83a5-185a383fb1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:51.170150 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1335/2685] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:51.180834 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b1e92-229a-415d-afc2-0044fab76baf admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:51.182467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-5f1b1e92-229a-415d-afc2-0044fab76baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:51.250328 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b1e92-229a-415d-afc2-0044fab76baf admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:51.254280 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db31356-6995-4c5d-ac67-83dcb60e4eea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:51.254947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db31356-6995-4c5d-ac67-83dcb60e4eea service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:51.255235 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db31356-6995-4c5d-ac67-83dcb60e4eea service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:51.255518 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db31356-6995-4c5d-ac67-83dcb60e4eea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:51.265193 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db31356-6995-4c5d-ac67-83dcb60e4eea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:51.267467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db31356-6995-4c5d-ac67-83dcb60e4eea service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:51.277398 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1351/2686] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:51.569342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4fb47-bb43-4c32-83a5-185a383fb1fa None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:51.570980 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b4d4fb47-bb43-4c32-83a5-185a383fb1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:51.587217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4fb47-bb43-4c32-83a5-185a383fb1fa None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:51.592196 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a6c19-e50c-42b2-a35f-a0b60626ae22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:51.593087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a6c19-e50c-42b2-a35f-a0b60626ae22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:51.593402 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a6c19-e50c-42b2-a35f-a0b60626ae22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:51.593684 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a6c19-e50c-42b2-a35f-a0b60626ae22 admin admin] PATH_INFO: `/v3/projects/37df6c266bac4dce818dc346255b53d2` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:51.595364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a6c19-e50c-42b2-a35f-a0b60626ae22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37df6c266bac4dce818dc346255b53d2)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:51.597392 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a6c19-e50c-42b2-a35f-a0b60626ae22 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:51.655676 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-493a6c19-e50c-42b2-a35f-a0b60626ae22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37df6c266bac4dce818dc346255b53d2', 'request_id': 'req-493a6c19-e50c-42b2-a35f-a0b60626ae22'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:51.667232 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-493a6c19-e50c-42b2-a35f-a0b60626ae22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37df6c266bac4dce818dc346255b53d2', 'request_id': 'req-493a6c19-e50c-42b2-a35f-a0b60626ae22'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:51.676095 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-493a6c19-e50c-42b2-a35f-a0b60626ae22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37df6c266bac4dce818dc346255b53d2', 'request_id': 'req-493a6c19-e50c-42b2-a35f-a0b60626ae22'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:51.687017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-493a6c19-e50c-42b2-a35f-a0b60626ae22 admin admin] The token cache is being invalidate because project 37df6c266bac4dce818dc346255b53d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:41:51.687318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-493a6c19-e50c-42b2-a35f-a0b60626ae22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:51.688232 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-493a6c19-e50c-42b2-a35f-a0b60626ae22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:51.690324 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1336/2687] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:41:51 2022] DELETE /identity/v3/projects/37df6c266bac4dce818dc346255b53d2 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:51.695807 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9207d-47b0-470b-87c0-1934d1e481ee None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:51.696490 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9207d-47b0-470b-87c0-1934d1e481ee None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:51.696490 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9207d-47b0-470b-87c0-1934d1e481ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:51.716087 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-2ab9207d-47b0-470b-87c0-1934d1e481ee None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:51.717375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-2ab9207d-47b0-470b-87c0-1934d1e481ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:51.785254 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1352/2688] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:41:51.793231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c4f08-3abc-4459-84b4-9f09eeab3a78 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:51.793765 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c4f08-3abc-4459-84b4-9f09eeab3a78 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:51.793984 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c4f08-3abc-4459-84b4-9f09eeab3a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:51.815420 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-345c4f08-3abc-4459-84b4-9f09eeab3a78 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:51.816679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-345c4f08-3abc-4459-84b4-9f09eeab3a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:51.825130 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1337/2689] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:51.831022 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9207d-47b0-470b-87c0-1934d1e481ee None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:51.841454 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9207d-47b0-470b-87c0-1934d1e481ee None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:51.846777 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa33e6-07d0-47bf-8067-ce63e9137209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:51.847457 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa33e6-07d0-47bf-8067-ce63e9137209 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:51.847750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa33e6-07d0-47bf-8067-ce63e9137209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:51.848033 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa33e6-07d0-47bf-8067-ce63e9137209 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:51.848781 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa33e6-07d0-47bf-8067-ce63e9137209 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:41:51.849100 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa33e6-07d0-47bf-8067-ce63e9137209 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:51.850766 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa33e6-07d0-47bf-8067-ce63e9137209 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:51.865882 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1353/2690] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:41:51.870586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c4f08-3abc-4459-84b4-9f09eeab3a78 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:51.877202 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:51.881507 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c4f08-3abc-4459-84b4-9f09eeab3a78 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:51.887519 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c5fe3-b687-4e87-b1d8-d4dbc7d892d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:51.888266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c5fe3-b687-4e87-b1d8-d4dbc7d892d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:51.888748 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c5fe3-b687-4e87-b1d8-d4dbc7d892d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:51.889090 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c5fe3-b687-4e87-b1d8-d4dbc7d892d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:51.890052 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c5fe3-b687-4e87-b1d8-d4dbc7d892d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:51.893007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c5fe3-b687-4e87-b1d8-d4dbc7d892d4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:51.924535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1338/2691] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:41:51 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:51.928768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa33e6-07d0-47bf-8067-ce63e9137209 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:51.939916 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa33e6-07d0-47bf-8067-ce63e9137209 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:51.949404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:52.014302 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015fce3-c0dd-45b3-a0ca-ffa808c503ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:52.015007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015fce3-c0dd-45b3-a0ca-ffa808c503ec admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:52.015405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015fce3-c0dd-45b3-a0ca-ffa808c503ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:52.015696 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015fce3-c0dd-45b3-a0ca-ffa808c503ec admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:52.016473 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c015fce3-c0dd-45b3-a0ca-ffa808c503ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:52.018597 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c015fce3-c0dd-45b3-a0ca-ffa808c503ec admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:52.041498 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1354/2692] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:41:51 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:52.045468 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c5fe3-b687-4e87-b1d8-d4dbc7d892d4 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:52.057785 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c5fe3-b687-4e87-b1d8-d4dbc7d892d4 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:52.063682 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c111db3-66bb-4e28-b89c-6b0e944d238c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:52.064485 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c111db3-66bb-4e28-b89c-6b0e944d238c admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:52.064924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c111db3-66bb-4e28-b89c-6b0e944d238c admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:52.065254 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c111db3-66bb-4e28-b89c-6b0e944d238c admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:52.066005 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c111db3-66bb-4e28-b89c-6b0e944d238c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:52.067610 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c111db3-66bb-4e28-b89c-6b0e944d238c admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:52.080828 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1339/2693] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:41:52.085363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015fce3-c0dd-45b3-a0ca-ffa808c503ec admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:52.096557 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015fce3-c0dd-45b3-a0ca-ffa808c503ec admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:52.101381 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d395e203-3c6f-454a-8771-1ca633d9bc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:52.102058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395e203-3c6f-454a-8771-1ca633d9bc54 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:52.102346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395e203-3c6f-454a-8771-1ca633d9bc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:52.102628 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395e203-3c6f-454a-8771-1ca633d9bc54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:52.103362 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d395e203-3c6f-454a-8771-1ca633d9bc54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:52.104950 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d395e203-3c6f-454a-8771-1ca633d9bc54 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:52.122717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1355/2694] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:52.122860 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c111db3-66bb-4e28-b89c-6b0e944d238c admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:52.131547 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c111db3-66bb-4e28-b89c-6b0e944d238c admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:52.136283 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd584c24-16cc-48d2-9777-c7e005df22bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:52.137363 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd584c24-16cc-48d2-9777-c7e005df22bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:52.137549 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd584c24-16cc-48d2-9777-c7e005df22bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:52.137767 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd584c24-16cc-48d2-9777-c7e005df22bb admin admin] PATH_INFO: `/v3/projects/990754d5bad1424fae09f7b03b638bdb/users/d1aab7e1cabd48e49b7df7adfe7fd656/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:52.153803 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd584c24-16cc-48d2-9777-c7e005df22bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=990754d5bad1424fae09f7b03b638bdb, user_id=d1aab7e1cabd48e49b7df7adfe7fd656, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:52.157248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd584c24-16cc-48d2-9777-c7e005df22bb admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:52.171401 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1340/2695] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:52 2022] PUT /identity/v3/projects/990754d5bad1424fae09f7b03b638bdb/users/d1aab7e1cabd48e49b7df7adfe7fd656/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:41:52.182624 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069cc11-1987-4d2a-ae2b-76a24977f17d None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:52.182917 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069cc11-1987-4d2a-ae2b-76a24977f17d None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:52.183353 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069cc11-1987-4d2a-ae2b-76a24977f17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:52.214248 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-4069cc11-1987-4d2a-ae2b-76a24977f17d None None] MFA Rules not processed for user `d1aab7e1cabd48e49b7df7adfe7fd656`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:52.215596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4069cc11-1987-4d2a-ae2b-76a24977f17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:52.301494 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1356/2696] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:52.303719 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd584c24-16cc-48d2-9777-c7e005df22bb admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:52.305462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fd584c24-16cc-48d2-9777-c7e005df22bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:52.307831 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069cc11-1987-4d2a-ae2b-76a24977f17d None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:52.309263 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4069cc11-1987-4d2a-ae2b-76a24977f17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:52.368568 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd584c24-16cc-48d2-9777-c7e005df22bb admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:52.373040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8e29b-2394-48db-ab3c-4c6160e9452d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:52.373748 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8e29b-2394-48db-ab3c-4c6160e9452d service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:52.374031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8e29b-2394-48db-ab3c-4c6160e9452d service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:52.374313 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8e29b-2394-48db-ab3c-4c6160e9452d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:52.378407 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069cc11-1987-4d2a-ae2b-76a24977f17d None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:52.382140 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47246b45-64ed-4c6f-9c31-1032f334912b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:52.382806 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47246b45-64ed-4c6f-9c31-1032f334912b service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:52.383099 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47246b45-64ed-4c6f-9c31-1032f334912b service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:52.383383 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47246b45-64ed-4c6f-9c31-1032f334912b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:52.384518 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8e29b-2394-48db-ab3c-4c6160e9452d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:52.385433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-47246b45-64ed-4c6f-9c31-1032f334912b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:52.386002 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8e29b-2394-48db-ab3c-4c6160e9452d service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:52.396417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1341/2697] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:52.415965 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8e29b-2394-48db-ab3c-4c6160e9452d service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:52.441864 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:52.475324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47246b45-64ed-4c6f-9c31-1032f334912b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:52.476793 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47246b45-64ed-4c6f-9c31-1032f334912b service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:52.485647 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8e29b-2394-48db-ab3c-4c6160e9452d service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:52.489177 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1357/2698] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:41:52.490374 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b311f-4908-4d18-9bad-d9d4c99e1962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:52.491321 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b311f-4908-4d18-9bad-d9d4c99e1962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:52.491608 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b311f-4908-4d18-9bad-d9d4c99e1962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:52.491907 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b311f-4908-4d18-9bad-d9d4c99e1962 admin admin] PATH_INFO: `/v3/users/d1aab7e1cabd48e49b7df7adfe7fd656` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:52.493130 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b311f-4908-4d18-9bad-d9d4c99e1962 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1aab7e1cabd48e49b7df7adfe7fd656)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:52.495165 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b311f-4908-4d18-9bad-d9d4c99e1962 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:52.550431 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a21b311f-4908-4d18-9bad-d9d4c99e1962 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1aab7e1cabd48e49b7df7adfe7fd656'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:52.556163 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:52.568045 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a21b311f-4908-4d18-9bad-d9d4c99e1962 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1aab7e1cabd48e49b7df7adfe7fd656'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:52.568899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a21b311f-4908-4d18-9bad-d9d4c99e1962 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1aab7e1cabd48e49b7df7adfe7fd656'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:52.576752 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a21b311f-4908-4d18-9bad-d9d4c99e1962 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1aab7e1cabd48e49b7df7adfe7fd656'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:52.585980 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-a21b311f-4908-4d18-9bad-d9d4c99e1962 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1aab7e1cabd48e49b7df7adfe7fd656'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:52.593565 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1342/2699] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:41:52 2022] DELETE /identity/v3/users/d1aab7e1cabd48e49b7df7adfe7fd656 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:41:52.598393 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6db9a-c283-4732-b3b8-b85957a1b33b None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:52.598813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6db9a-c283-4732-b3b8-b85957a1b33b None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:52.599330 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6db9a-c283-4732-b3b8-b85957a1b33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:52.618750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-65b6db9a-c283-4732-b3b8-b85957a1b33b None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:52.620071 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-65b6db9a-c283-4732-b3b8-b85957a1b33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:52.689006 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1358/2700] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:52.699552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b311f-4908-4d18-9bad-d9d4c99e1962 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:52.701024 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-a21b311f-4908-4d18-9bad-d9d4c99e1962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:52.769629 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b311f-4908-4d18-9bad-d9d4c99e1962 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:52.773304 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db557dc-969c-4a39-be6f-775e088744c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:52.773967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db557dc-969c-4a39-be6f-775e088744c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:52.774257 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db557dc-969c-4a39-be6f-775e088744c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:52.774542 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db557dc-969c-4a39-be6f-775e088744c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:52.786249 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db557dc-969c-4a39-be6f-775e088744c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:52.787857 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db557dc-969c-4a39-be6f-775e088744c2 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:52.801301 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1343/2701] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:53.166606 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6db9a-c283-4732-b3b8-b85957a1b33b None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:53.168428 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-65b6db9a-c283-4732-b3b8-b85957a1b33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:53.184963 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6db9a-c283-4732-b3b8-b85957a1b33b None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:53.192070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f1ac9-3330-4987-9eb7-b57275e206ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:53.193115 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f1ac9-3330-4987-9eb7-b57275e206ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:53.193875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f1ac9-3330-4987-9eb7-b57275e206ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:53.193875 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f1ac9-3330-4987-9eb7-b57275e206ad admin admin] PATH_INFO: `/v3/projects/990754d5bad1424fae09f7b03b638bdb` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:53.195257 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f1ac9-3330-4987-9eb7-b57275e206ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=990754d5bad1424fae09f7b03b638bdb)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:53.197326 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f1ac9-3330-4987-9eb7-b57275e206ad admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:53.265030 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-f20f1ac9-3330-4987-9eb7-b57275e206ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '990754d5bad1424fae09f7b03b638bdb', 'request_id': 'req-f20f1ac9-3330-4987-9eb7-b57275e206ad'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:53.272865 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-f20f1ac9-3330-4987-9eb7-b57275e206ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '990754d5bad1424fae09f7b03b638bdb', 'request_id': 'req-f20f1ac9-3330-4987-9eb7-b57275e206ad'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:53.280039 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-f20f1ac9-3330-4987-9eb7-b57275e206ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '990754d5bad1424fae09f7b03b638bdb', 'request_id': 'req-f20f1ac9-3330-4987-9eb7-b57275e206ad'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:53.295998 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-f20f1ac9-3330-4987-9eb7-b57275e206ad admin admin] The token cache is being invalidate because project 990754d5bad1424fae09f7b03b638bdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:41:53.296306 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-f20f1ac9-3330-4987-9eb7-b57275e206ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:53.297716 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-f20f1ac9-3330-4987-9eb7-b57275e206ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:41:53.302861 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1359/2702] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:41:53 2022] DELETE /identity/v3/projects/990754d5bad1424fae09f7b03b638bdb => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:41:53.305281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525b0c2-10da-482e-b68a-e492e09ad614 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:53.305835 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525b0c2-10da-482e-b68a-e492e09ad614 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:53.306409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525b0c2-10da-482e-b68a-e492e09ad614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:53.326704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4525b0c2-10da-482e-b68a-e492e09ad614 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:53.327963 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4525b0c2-10da-482e-b68a-e492e09ad614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:53.396380 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1344/2703] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:53.404160 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419cc365-2db6-44b5-b3b1-075c5f8f7821 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:53.404467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419cc365-2db6-44b5-b3b1-075c5f8f7821 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:53.404761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419cc365-2db6-44b5-b3b1-075c5f8f7821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:53.425912 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-419cc365-2db6-44b5-b3b1-075c5f8f7821 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:53.427153 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-419cc365-2db6-44b5-b3b1-075c5f8f7821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:53.435711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1360/2704] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:53.440786 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525b0c2-10da-482e-b68a-e492e09ad614 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:53.450716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525b0c2-10da-482e-b68a-e492e09ad614 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:53.457119 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4709dc-2861-4301-8254-9da6ceb716d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:53.457847 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4709dc-2861-4301-8254-9da6ceb716d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:53.458138 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4709dc-2861-4301-8254-9da6ceb716d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:53.458420 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4709dc-2861-4301-8254-9da6ceb716d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:53.459170 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4709dc-2861-4301-8254-9da6ceb716d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:41:53.459485 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4709dc-2861-4301-8254-9da6ceb716d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:53.461205 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4709dc-2861-4301-8254-9da6ceb716d8 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:53.478659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1345/2705] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:41:53.480118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419cc365-2db6-44b5-b3b1-075c5f8f7821 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:53.490570 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419cc365-2db6-44b5-b3b1-075c5f8f7821 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:53.495440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c7e38-9404-4561-8954-fe1fc0f59d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:53.496127 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c7e38-9404-4561-8954-fe1fc0f59d1b admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:53.496693 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c7e38-9404-4561-8954-fe1fc0f59d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:53.496693 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c7e38-9404-4561-8954-fe1fc0f59d1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:53.497731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c7e38-9404-4561-8954-fe1fc0f59d1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:53.499680 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c7e38-9404-4561-8954-fe1fc0f59d1b admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:53.540458 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1361/2706] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:41:53 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:53.544502 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4709dc-2861-4301-8254-9da6ceb716d8 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:53.554803 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4709dc-2861-4301-8254-9da6ceb716d8 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:53.617246 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37811fca-c5c0-4465-a499-2f63ddfd1e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:53.617928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37811fca-c5c0-4465-a499-2f63ddfd1e73 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:53.618218 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37811fca-c5c0-4465-a499-2f63ddfd1e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:53.618491 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37811fca-c5c0-4465-a499-2f63ddfd1e73 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:53.619201 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37811fca-c5c0-4465-a499-2f63ddfd1e73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:53.621267 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37811fca-c5c0-4465-a499-2f63ddfd1e73 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:53.628736 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:53.647825 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1346/2707] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:41:53 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:41:53.649291 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c7e38-9404-4561-8954-fe1fc0f59d1b admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:53.661759 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c7e38-9404-4561-8954-fe1fc0f59d1b admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:53.666715 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ff864-1036-4855-8443-7d549901a25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:53.667511 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ff864-1036-4855-8443-7d549901a25a admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:53.668017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ff864-1036-4855-8443-7d549901a25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:53.668654 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ff864-1036-4855-8443-7d549901a25a admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:53.669268 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ff864-1036-4855-8443-7d549901a25a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:53.670973 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ff864-1036-4855-8443-7d549901a25a admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:53.685807 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1362/2708] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:53.687074 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37811fca-c5c0-4465-a499-2f63ddfd1e73 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:53.698005 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37811fca-c5c0-4465-a499-2f63ddfd1e73 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:53.704615 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fecffed-d13b-4b94-a6a9-b58ac1088fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:53.705337 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecffed-d13b-4b94-a6a9-b58ac1088fec admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:53.705929 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecffed-d13b-4b94-a6a9-b58ac1088fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:53.705929 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecffed-d13b-4b94-a6a9-b58ac1088fec admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:53.706657 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fecffed-d13b-4b94-a6a9-b58ac1088fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:53.708263 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fecffed-d13b-4b94-a6a9-b58ac1088fec admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:53.721041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1347/2709] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:41:53.725195 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ff864-1036-4855-8443-7d549901a25a admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:53.736916 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ff864-1036-4855-8443-7d549901a25a admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:53.741701 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01709083-0b43-4c8e-9073-bf0564cc818f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:53.742601 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01709083-0b43-4c8e-9073-bf0564cc818f admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:53.743042 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01709083-0b43-4c8e-9073-bf0564cc818f admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:53.743327 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01709083-0b43-4c8e-9073-bf0564cc818f admin admin] PATH_INFO: `/v3/projects/554c281060a446ed9b2d02bad7b8ec1d/users/9de9e460242c4a02ad28af485ad851e5/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:53.757467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:53.761095 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01709083-0b43-4c8e-9073-bf0564cc818f admin admin] RBAC: Authorizing `identity:create_grant(project_id=554c281060a446ed9b2d02bad7b8ec1d, user_id=9de9e460242c4a02ad28af485ad851e5, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:53.764499 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01709083-0b43-4c8e-9073-bf0564cc818f admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:53.779861 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1363/2710] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:41:53 2022] PUT /identity/v3/projects/554c281060a446ed9b2d02bad7b8ec1d/users/9de9e460242c4a02ad28af485ad851e5/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:41:53.785268 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee41cb3-46a6-4baf-9bb7-ae466dc63dd2 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:53.785569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee41cb3-46a6-4baf-9bb7-ae466dc63dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:53.785857 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee41cb3-46a6-4baf-9bb7-ae466dc63dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:53.804681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-fee41cb3-46a6-4baf-9bb7-ae466dc63dd2 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:53.805979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fee41cb3-46a6-4baf-9bb7-ae466dc63dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:53.877112 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1348/2711] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:53.887928 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01709083-0b43-4c8e-9073-bf0564cc818f admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:53.889795 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-01709083-0b43-4c8e-9073-bf0564cc818f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:53.958533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01709083-0b43-4c8e-9073-bf0564cc818f admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:53.963191 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cfae22-219a-4cad-9bdf-593eb5b61cbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:53.963974 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfae22-219a-4cad-9bdf-593eb5b61cbe service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:53.964342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfae22-219a-4cad-9bdf-593eb5b61cbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:53.964701 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfae22-219a-4cad-9bdf-593eb5b61cbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:53.979513 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cfae22-219a-4cad-9bdf-593eb5b61cbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:53.979513 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cfae22-219a-4cad-9bdf-593eb5b61cbe service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:53.992362 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1364/2712] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:54.865012 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957315f-5c83-4e5d-9500-361250c8f6ac None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:54.868223 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957315f-5c83-4e5d-9500-361250c8f6ac None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:54.868223 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957315f-5c83-4e5d-9500-361250c8f6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:54.898108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4957315f-5c83-4e5d-9500-361250c8f6ac None None] MFA Rules not processed for user `9de9e460242c4a02ad28af485ad851e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:54.899697 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4957315f-5c83-4e5d-9500-361250c8f6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:54.993795 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1349/2713] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:55.010202 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cfae22-219a-4cad-9bdf-593eb5b61cbe service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:55.011877 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-70cfae22-219a-4cad-9bdf-593eb5b61cbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:55.089286 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cfae22-219a-4cad-9bdf-593eb5b61cbe service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:55.093624 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd43a0b-ecd7-4335-8f8b-5f254abe15da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:55.095102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd43a0b-ecd7-4335-8f8b-5f254abe15da service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:55.095590 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd43a0b-ecd7-4335-8f8b-5f254abe15da service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:55.095941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd43a0b-ecd7-4335-8f8b-5f254abe15da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:55.107908 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd43a0b-ecd7-4335-8f8b-5f254abe15da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:55.110168 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd43a0b-ecd7-4335-8f8b-5f254abe15da service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:55.120960 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1365/2714] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:57.708496 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357f8aa-f6fa-49d1-b7ee-44b545cc0773 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:57.708496 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357f8aa-f6fa-49d1-b7ee-44b545cc0773 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:57.709300 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357f8aa-f6fa-49d1-b7ee-44b545cc0773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:57.731655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-6357f8aa-f6fa-49d1-b7ee-44b545cc0773 None None] MFA Rules not processed for user `9de9e460242c4a02ad28af485ad851e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:57.732809 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6357f8aa-f6fa-49d1-b7ee-44b545cc0773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:57.745610 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1350/2715] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:41:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:04.338395 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd43a0b-ecd7-4335-8f8b-5f254abe15da service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:04.340096 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4bd43a0b-ecd7-4335-8f8b-5f254abe15da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:04.437554 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd43a0b-ecd7-4335-8f8b-5f254abe15da service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:04.443320 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360513e-2ab6-4fff-aa29-d1b202a262e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689a31e26c6b454a91c3fd79678b4d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:04.444018 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360513e-2ab6-4fff-aa29-d1b202a262e9 service nova] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:04.444312 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360513e-2ab6-4fff-aa29-d1b202a262e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:04.444599 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360513e-2ab6-4fff-aa29-d1b202a262e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:04.447393 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-2360513e-2ab6-4fff-aa29-d1b202a262e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:04.492992 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:04.530230 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360513e-2ab6-4fff-aa29-d1b202a262e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:04.532795 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360513e-2ab6-4fff-aa29-d1b202a262e9 service nova] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:04.545046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1366/2716] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:42:07.722463 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:08.574558 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357f8aa-f6fa-49d1-b7ee-44b545cc0773 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:08.576012 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6357f8aa-f6fa-49d1-b7ee-44b545cc0773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:08.590276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357f8aa-f6fa-49d1-b7ee-44b545cc0773 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:08.595875 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172b7b9-c938-4fd0-9ccc-91b696fd5d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:08.596654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172b7b9-c938-4fd0-9ccc-91b696fd5d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:08.596943 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172b7b9-c938-4fd0-9ccc-91b696fd5d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:08.597337 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172b7b9-c938-4fd0-9ccc-91b696fd5d05 admin admin] PATH_INFO: `/v3/users/1233ca8155fe4be3a6d805299e5143fd` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:08.600277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172b7b9-c938-4fd0-9ccc-91b696fd5d05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1233ca8155fe4be3a6d805299e5143fd)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:08.603505 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172b7b9-c938-4fd0-9ccc-91b696fd5d05 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:08.673455 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e172b7b9-c938-4fd0-9ccc-91b696fd5d05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1233ca8155fe4be3a6d805299e5143fd'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:08.695939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e172b7b9-c938-4fd0-9ccc-91b696fd5d05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1233ca8155fe4be3a6d805299e5143fd'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:08.696741 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e172b7b9-c938-4fd0-9ccc-91b696fd5d05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1233ca8155fe4be3a6d805299e5143fd'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:08.705604 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e172b7b9-c938-4fd0-9ccc-91b696fd5d05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1233ca8155fe4be3a6d805299e5143fd'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:08.715414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e172b7b9-c938-4fd0-9ccc-91b696fd5d05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1233ca8155fe4be3a6d805299e5143fd'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:08.726021 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1351/2717] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:42:08 2022] DELETE /identity/v3/users/1233ca8155fe4be3a6d805299e5143fd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:08.731859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fa5fc-bee7-4117-b888-fc3c25f25a49 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:08.732556 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fa5fc-bee7-4117-b888-fc3c25f25a49 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:08.732831 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fa5fc-bee7-4117-b888-fc3c25f25a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:08.757314 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-4e1fa5fc-bee7-4117-b888-fc3c25f25a49 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:08.758502 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4e1fa5fc-bee7-4117-b888-fc3c25f25a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:08.861734 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1367/2718] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:08.875871 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172b7b9-c938-4fd0-9ccc-91b696fd5d05 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:08.876830 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1fa5fc-bee7-4117-b888-fc3c25f25a49 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:08.877389 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e172b7b9-c938-4fd0-9ccc-91b696fd5d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:08.878418 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4e1fa5fc-bee7-4117-b888-fc3c25f25a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:08.949128 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172b7b9-c938-4fd0-9ccc-91b696fd5d05 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:08.953994 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9cbbb-6137-446d-8ea3-1a6d61386a79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:08.954244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1fa5fc-bee7-4117-b888-fc3c25f25a49 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:08.954668 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9cbbb-6137-446d-8ea3-1a6d61386a79 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:08.954962 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9cbbb-6137-446d-8ea3-1a6d61386a79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:08.955243 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9cbbb-6137-446d-8ea3-1a6d61386a79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:08.958179 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67435991-95d3-491a-b964-547cb2c71a34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:08.958904 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67435991-95d3-491a-b964-547cb2c71a34 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:08.959258 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67435991-95d3-491a-b964-547cb2c71a34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:08.959615 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67435991-95d3-491a-b964-547cb2c71a34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:08.961535 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-67435991-95d3-491a-b964-547cb2c71a34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:08.965508 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9cbbb-6137-446d-8ea3-1a6d61386a79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:08.967076 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9cbbb-6137-446d-8ea3-1a6d61386a79 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:08.975114 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67435991-95d3-491a-b964-547cb2c71a34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:08.976591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67435991-95d3-491a-b964-547cb2c71a34 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:08.977983 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1352/2719] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:42:08.988189 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1368/2720] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:42:09.461289 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9cbbb-6137-446d-8ea3-1a6d61386a79 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:09.463781 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-bfc9cbbb-6137-446d-8ea3-1a6d61386a79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:09.490091 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9cbbb-6137-446d-8ea3-1a6d61386a79 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:09.495665 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357460f-d80c-498b-ad15-0aece301bd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:09.496535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357460f-d80c-498b-ad15-0aece301bd26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:09.496901 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357460f-d80c-498b-ad15-0aece301bd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:09.497217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357460f-d80c-498b-ad15-0aece301bd26 admin admin] PATH_INFO: `/v3/projects/706c5aa484c8430ca38159cca5d591d2` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:09.498542 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357460f-d80c-498b-ad15-0aece301bd26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=706c5aa484c8430ca38159cca5d591d2)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:09.501348 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357460f-d80c-498b-ad15-0aece301bd26 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:09.565857 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4357460f-d80c-498b-ad15-0aece301bd26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '706c5aa484c8430ca38159cca5d591d2', 'request_id': 'req-4357460f-d80c-498b-ad15-0aece301bd26'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:09.575561 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4357460f-d80c-498b-ad15-0aece301bd26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '706c5aa484c8430ca38159cca5d591d2', 'request_id': 'req-4357460f-d80c-498b-ad15-0aece301bd26'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:09.584084 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4357460f-d80c-498b-ad15-0aece301bd26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '706c5aa484c8430ca38159cca5d591d2', 'request_id': 'req-4357460f-d80c-498b-ad15-0aece301bd26'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:09.594217 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:09.601685 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4357460f-d80c-498b-ad15-0aece301bd26 admin admin] The token cache is being invalidate because project 706c5aa484c8430ca38159cca5d591d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:42:09.602447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4357460f-d80c-498b-ad15-0aece301bd26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:09.603791 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4357460f-d80c-498b-ad15-0aece301bd26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:09.607416 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1353/2721] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:42:09 2022] DELETE /identity/v3/projects/706c5aa484c8430ca38159cca5d591d2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:09.612629 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0232605-4291-4e5d-81b0-02c6a8a94b52 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:09.612980 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0232605-4291-4e5d-81b0-02c6a8a94b52 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:09.613367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0232605-4291-4e5d-81b0-02c6a8a94b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:09.646271 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-e0232605-4291-4e5d-81b0-02c6a8a94b52 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:09.647904 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-e0232605-4291-4e5d-81b0-02c6a8a94b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:09.730783 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1369/2722] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:09.739104 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ae35d-8749-4599-b94d-821263e411e1 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:09.739657 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ae35d-8749-4599-b94d-821263e411e1 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:09.739913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ae35d-8749-4599-b94d-821263e411e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:09.761657 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-055ae35d-8749-4599-b94d-821263e411e1 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:09.763035 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-055ae35d-8749-4599-b94d-821263e411e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:09.772809 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1354/2723] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:09.777087 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0232605-4291-4e5d-81b0-02c6a8a94b52 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:09.790308 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0232605-4291-4e5d-81b0-02c6a8a94b52 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:09.795460 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70fbbfc-8d2b-4007-90a7-3bb5aaae9083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:09.796774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fbbfc-8d2b-4007-90a7-3bb5aaae9083 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:09.797146 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fbbfc-8d2b-4007-90a7-3bb5aaae9083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:09.797621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fbbfc-8d2b-4007-90a7-3bb5aaae9083 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:09.798451 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70fbbfc-8d2b-4007-90a7-3bb5aaae9083 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:42:09.798844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70fbbfc-8d2b-4007-90a7-3bb5aaae9083 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:09.800547 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70fbbfc-8d2b-4007-90a7-3bb5aaae9083 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:09.819324 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1370/2724] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:42:09.823631 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ae35d-8749-4599-b94d-821263e411e1 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:09.833841 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ae35d-8749-4599-b94d-821263e411e1 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:09.838699 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f12997-bce6-4bf7-b48f-baf624ec7760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:09.839392 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f12997-bce6-4bf7-b48f-baf624ec7760 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:09.839684 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f12997-bce6-4bf7-b48f-baf624ec7760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:09.839963 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f12997-bce6-4bf7-b48f-baf624ec7760 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:09.840697 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f12997-bce6-4bf7-b48f-baf624ec7760 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:09.842678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f12997-bce6-4bf7-b48f-baf624ec7760 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:09.877141 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1355/2725] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:42:09 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:42:09.878386 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70fbbfc-8d2b-4007-90a7-3bb5aaae9083 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:09.885325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:09.890557 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70fbbfc-8d2b-4007-90a7-3bb5aaae9083 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:09.952360 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c290d7-fece-4bdf-b3f1-7eaf786755f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:09.953677 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c290d7-fece-4bdf-b3f1-7eaf786755f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:09.953677 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c290d7-fece-4bdf-b3f1-7eaf786755f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:09.953894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c290d7-fece-4bdf-b3f1-7eaf786755f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:09.954481 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c290d7-fece-4bdf-b3f1-7eaf786755f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:09.956471 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c290d7-fece-4bdf-b3f1-7eaf786755f3 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:09.979281 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1371/2726] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:42:09 2022] POST /identity/v3/users => generated 284 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:42:09.985266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f12997-bce6-4bf7-b48f-baf624ec7760 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:09.997908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f12997-bce6-4bf7-b48f-baf624ec7760 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:10.004291 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ff321-bfd0-4846-9f6e-db75647deb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:10.005133 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ff321-bfd0-4846-9f6e-db75647deb49 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:10.005586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ff321-bfd0-4846-9f6e-db75647deb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:10.005868 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ff321-bfd0-4846-9f6e-db75647deb49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:10.006626 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ff321-bfd0-4846-9f6e-db75647deb49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:10.008286 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ff321-bfd0-4846-9f6e-db75647deb49 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:10.022030 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1356/2727] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:42:10.026058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c290d7-fece-4bdf-b3f1-7eaf786755f3 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:10.038100 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c290d7-fece-4bdf-b3f1-7eaf786755f3 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:10.042869 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55712711-91ef-44cf-878d-4eb6d420a893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:10.043568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55712711-91ef-44cf-878d-4eb6d420a893 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:10.043846 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55712711-91ef-44cf-878d-4eb6d420a893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:10.044134 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55712711-91ef-44cf-878d-4eb6d420a893 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:10.044891 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55712711-91ef-44cf-878d-4eb6d420a893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:10.047763 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55712711-91ef-44cf-878d-4eb6d420a893 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:10.064243 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1372/2728] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:42:10.068546 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ff321-bfd0-4846-9f6e-db75647deb49 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:10.086364 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ff321-bfd0-4846-9f6e-db75647deb49 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:10.092173 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782afab1-5274-4e36-a521-750a934d6a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:10.093182 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782afab1-5274-4e36-a521-750a934d6a9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:10.093532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782afab1-5274-4e36-a521-750a934d6a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:10.094036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782afab1-5274-4e36-a521-750a934d6a9d admin admin] PATH_INFO: `/v3/projects/039b7c844f5542f2a55b6d7e7be5aa96/users/d5de893fcbb341168261082757384420/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:10.111983 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782afab1-5274-4e36-a521-750a934d6a9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=039b7c844f5542f2a55b6d7e7be5aa96, user_id=d5de893fcbb341168261082757384420, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:10.114842 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782afab1-5274-4e36-a521-750a934d6a9d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:10.142682 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1357/2729] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:42:10 2022] PUT /identity/v3/projects/039b7c844f5542f2a55b6d7e7be5aa96/users/d5de893fcbb341168261082757384420/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:10.157741 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d58984-26e2-4a37-82e3-b043470de1a4 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:10.157741 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d58984-26e2-4a37-82e3-b043470de1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:10.157990 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d58984-26e2-4a37-82e3-b043470de1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:10.192846 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-55d58984-26e2-4a37-82e3-b043470de1a4 None None] MFA Rules not processed for user `d5de893fcbb341168261082757384420`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:10.194303 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-55d58984-26e2-4a37-82e3-b043470de1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:10.293188 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1373/2730] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:10 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:10.303166 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782afab1-5274-4e36-a521-750a934d6a9d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:10.305159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-782afab1-5274-4e36-a521-750a934d6a9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:10.400651 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782afab1-5274-4e36-a521-750a934d6a9d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:10.406803 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f237c8-9b16-4102-a6a7-158fdce80da1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '327b347d6295422c99ce7b44eab00467', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:10.407813 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f237c8-9b16-4102-a6a7-158fdce80da1 service glance] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:10.408236 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f237c8-9b16-4102-a6a7-158fdce80da1 service glance] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:10.408601 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f237c8-9b16-4102-a6a7-158fdce80da1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:10.418716 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f237c8-9b16-4102-a6a7-158fdce80da1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:10.420291 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f237c8-9b16-4102-a6a7-158fdce80da1 service glance] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:10.431634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1358/2731] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:10 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:42:13.069838 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d58984-26e2-4a37-82e3-b043470de1a4 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:13.071979 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-55d58984-26e2-4a37-82e3-b043470de1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:13.167414 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d58984-26e2-4a37-82e3-b043470de1a4 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:13.173764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd33ed9-fb98-4a72-9b9e-dfe60a0eafa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:13.174769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd33ed9-fb98-4a72-9b9e-dfe60a0eafa8 service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:13.174769 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd33ed9-fb98-4a72-9b9e-dfe60a0eafa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:13.175111 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd33ed9-fb98-4a72-9b9e-dfe60a0eafa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:13.177144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-dbd33ed9-fb98-4a72-9b9e-dfe60a0eafa8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:13.221512 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:13.288266 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd33ed9-fb98-4a72-9b9e-dfe60a0eafa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:13.289894 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd33ed9-fb98-4a72-9b9e-dfe60a0eafa8 service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:13.300766 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1374/2732] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:42:16.812055 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f237c8-9b16-4102-a6a7-158fdce80da1 service glance] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:16.814744 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-43f237c8-9b16-4102-a6a7-158fdce80da1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:16.926570 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f237c8-9b16-4102-a6a7-158fdce80da1 service glance] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:16.932150 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33880c6c-6953-4788-b3ca-61cebef17fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:16.933347 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33880c6c-6953-4788-b3ca-61cebef17fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:16.933646 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33880c6c-6953-4788-b3ca-61cebef17fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:16.933936 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33880c6c-6953-4788-b3ca-61cebef17fc4 admin admin] PATH_INFO: `/v3/users/b41334c5088e4f4990637fd7e2964999` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:16.935291 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33880c6c-6953-4788-b3ca-61cebef17fc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b41334c5088e4f4990637fd7e2964999)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:16.937877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33880c6c-6953-4788-b3ca-61cebef17fc4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:17.004818 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:17.020287 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-33880c6c-6953-4788-b3ca-61cebef17fc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b41334c5088e4f4990637fd7e2964999'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:17.038928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-33880c6c-6953-4788-b3ca-61cebef17fc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b41334c5088e4f4990637fd7e2964999'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:17.040987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-33880c6c-6953-4788-b3ca-61cebef17fc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b41334c5088e4f4990637fd7e2964999'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:17.058562 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-33880c6c-6953-4788-b3ca-61cebef17fc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b41334c5088e4f4990637fd7e2964999'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:17.072741 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-33880c6c-6953-4788-b3ca-61cebef17fc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b41334c5088e4f4990637fd7e2964999'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:17.083268 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1359/2733] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:42:16 2022] DELETE /identity/v3/users/b41334c5088e4f4990637fd7e2964999 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:42:17.323153 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd33ed9-fb98-4a72-9b9e-dfe60a0eafa8 service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:17.378785 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd33ed9-fb98-4a72-9b9e-dfe60a0eafa8 service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:17.464747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baaaa61-ed94-4a57-824f-ae2a8845c4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:17.465693 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baaaa61-ed94-4a57-824f-ae2a8845c4f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:17.465985 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baaaa61-ed94-4a57-824f-ae2a8845c4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:17.466271 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baaaa61-ed94-4a57-824f-ae2a8845c4f0 admin admin] PATH_INFO: `/v3/projects/35fdc8ef703e43f1a730a868721c4a2d` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:17.468009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baaaa61-ed94-4a57-824f-ae2a8845c4f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35fdc8ef703e43f1a730a868721c4a2d)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:17.470011 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baaaa61-ed94-4a57-824f-ae2a8845c4f0 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:17.564870 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9baaaa61-ed94-4a57-824f-ae2a8845c4f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35fdc8ef703e43f1a730a868721c4a2d', 'request_id': 'req-9baaaa61-ed94-4a57-824f-ae2a8845c4f0'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:17.574433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9baaaa61-ed94-4a57-824f-ae2a8845c4f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35fdc8ef703e43f1a730a868721c4a2d', 'request_id': 'req-9baaaa61-ed94-4a57-824f-ae2a8845c4f0'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:17.583602 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9baaaa61-ed94-4a57-824f-ae2a8845c4f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35fdc8ef703e43f1a730a868721c4a2d', 'request_id': 'req-9baaaa61-ed94-4a57-824f-ae2a8845c4f0'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:17.602556 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9baaaa61-ed94-4a57-824f-ae2a8845c4f0 admin admin] The token cache is being invalidate because project 35fdc8ef703e43f1a730a868721c4a2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:42:17.602866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9baaaa61-ed94-4a57-824f-ae2a8845c4f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:17.603587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9baaaa61-ed94-4a57-824f-ae2a8845c4f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:17.605640 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1375/2734] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:42:17 2022] DELETE /identity/v3/projects/35fdc8ef703e43f1a730a868721c4a2d => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:17.611310 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33880c6c-6953-4788-b3ca-61cebef17fc4 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:17.614407 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-33880c6c-6953-4788-b3ca-61cebef17fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:17.636120 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33880c6c-6953-4788-b3ca-61cebef17fc4 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:17.751908 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e9fdb-e69f-406d-8cfe-3757f03e7aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:17.753122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e9fdb-e69f-406d-8cfe-3757f03e7aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:17.754206 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e9fdb-e69f-406d-8cfe-3757f03e7aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:17.757367 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e9fdb-e69f-406d-8cfe-3757f03e7aa2 admin admin] PATH_INFO: `/v3/users/1c021dd109504201be15e79f3c07378e` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:17.757367 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e9fdb-e69f-406d-8cfe-3757f03e7aa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c021dd109504201be15e79f3c07378e)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:17.759108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e9fdb-e69f-406d-8cfe-3757f03e7aa2 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:17.835027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f97e9fdb-e69f-406d-8cfe-3757f03e7aa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c021dd109504201be15e79f3c07378e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:17.852200 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f97e9fdb-e69f-406d-8cfe-3757f03e7aa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c021dd109504201be15e79f3c07378e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:17.853131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f97e9fdb-e69f-406d-8cfe-3757f03e7aa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c021dd109504201be15e79f3c07378e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:17.865511 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f97e9fdb-e69f-406d-8cfe-3757f03e7aa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c021dd109504201be15e79f3c07378e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:17.879575 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-f97e9fdb-e69f-406d-8cfe-3757f03e7aa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c021dd109504201be15e79f3c07378e'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:17.891546 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1360/2735] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:42:17 2022] DELETE /identity/v3/users/1c021dd109504201be15e79f3c07378e => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:18.149551 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baaaa61-ed94-4a57-824f-ae2a8845c4f0 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:18.152804 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-9baaaa61-ed94-4a57-824f-ae2a8845c4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:18.193168 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baaaa61-ed94-4a57-824f-ae2a8845c4f0 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:18.292278 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:18.312808 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f8ab0-36e7-4cb5-957f-60efd7be561e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:18.315414 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f8ab0-36e7-4cb5-957f-60efd7be561e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:18.315863 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f8ab0-36e7-4cb5-957f-60efd7be561e admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:18.316329 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f8ab0-36e7-4cb5-957f-60efd7be561e admin admin] PATH_INFO: `/v3/projects/119182b448df4b0982dcd69db60fd7a6` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:18.318315 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f8ab0-36e7-4cb5-957f-60efd7be561e admin admin] RBAC: Authorizing `identity:delete_project(project_id=119182b448df4b0982dcd69db60fd7a6)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:18.321313 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f8ab0-36e7-4cb5-957f-60efd7be561e admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:18.383794 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-252f8ab0-36e7-4cb5-957f-60efd7be561e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '119182b448df4b0982dcd69db60fd7a6', 'request_id': 'req-252f8ab0-36e7-4cb5-957f-60efd7be561e'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:18.393619 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-252f8ab0-36e7-4cb5-957f-60efd7be561e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '119182b448df4b0982dcd69db60fd7a6', 'request_id': 'req-252f8ab0-36e7-4cb5-957f-60efd7be561e'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:18.407214 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-252f8ab0-36e7-4cb5-957f-60efd7be561e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '119182b448df4b0982dcd69db60fd7a6', 'request_id': 'req-252f8ab0-36e7-4cb5-957f-60efd7be561e'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:18.430403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-252f8ab0-36e7-4cb5-957f-60efd7be561e admin admin] The token cache is being invalidate because project 119182b448df4b0982dcd69db60fd7a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:42:18.430713 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-252f8ab0-36e7-4cb5-957f-60efd7be561e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:18.431392 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-252f8ab0-36e7-4cb5-957f-60efd7be561e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:18.433035 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1376/2736] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:42:18 2022] DELETE /identity/v3/projects/119182b448df4b0982dcd69db60fd7a6 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:21.606580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb1009-d0df-4016-8b40-e6455ef92fe6 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:21.607153 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb1009-d0df-4016-8b40-e6455ef92fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:21.607153 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb1009-d0df-4016-8b40-e6455ef92fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:21.662626 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-7bdb1009-d0df-4016-8b40-e6455ef92fe6 None None] MFA Rules not processed for user `faf0a0102da24cedaedd0c67273e275b`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:21.663885 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7bdb1009-d0df-4016-8b40-e6455ef92fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:21.721127 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:21.759229 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1361/2737] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:42:23.556925 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f8ab0-36e7-4cb5-957f-60efd7be561e admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:23.559194 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-252f8ab0-36e7-4cb5-957f-60efd7be561e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:23.696997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f8ab0-36e7-4cb5-957f-60efd7be561e admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:23.708091 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f0e28-947b-42a3-9a87-42f494f53215 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75f0586a8c1941448d750918183f6dc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:23.709527 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f0e28-947b-42a3-9a87-42f494f53215 service placement] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:23.709818 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f0e28-947b-42a3-9a87-42f494f53215 service placement] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:23.710095 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f0e28-947b-42a3-9a87-42f494f53215 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:23.713772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-d59f0e28-947b-42a3-9a87-42f494f53215 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:23.730633 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f0e28-947b-42a3-9a87-42f494f53215 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:23.732828 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f0e28-947b-42a3-9a87-42f494f53215 service placement] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:23.765496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1377/2738] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:42:28.055057 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb1009-d0df-4016-8b40-e6455ef92fe6 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:28.056400 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7bdb1009-d0df-4016-8b40-e6455ef92fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:28.136556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb1009-d0df-4016-8b40-e6455ef92fe6 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:28.140100 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7253c-e498-400d-87a3-740405c9e0ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:28.140884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7253c-e498-400d-87a3-740405c9e0ef service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:28.142342 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7253c-e498-400d-87a3-740405c9e0ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:28.142634 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7253c-e498-400d-87a3-740405c9e0ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:28.144472 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-98a7253c-e498-400d-87a3-740405c9e0ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:28.208609 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7253c-e498-400d-87a3-740405c9e0ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:28.210114 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7253c-e498-400d-87a3-740405c9e0ef service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:28.221135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1362/2739] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:42:33.150532 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f0e28-947b-42a3-9a87-42f494f53215 service placement] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:33.150532 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-d59f0e28-947b-42a3-9a87-42f494f53215 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:33.164779 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f0e28-947b-42a3-9a87-42f494f53215 service placement] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:33.169994 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbc4fd-81e5-4707-8180-d3553494c0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:33.170938 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbc4fd-81e5-4707-8180-d3553494c0a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:33.171228 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbc4fd-81e5-4707-8180-d3553494c0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:33.171517 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbc4fd-81e5-4707-8180-d3553494c0a1 admin admin] PATH_INFO: `/v3/users/2fbaf6d894e64f19bf041e7be20a6242` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:33.172962 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbc4fd-81e5-4707-8180-d3553494c0a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fbaf6d894e64f19bf041e7be20a6242)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:33.175002 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbc4fd-81e5-4707-8180-d3553494c0a1 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:33.245648 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-86cbc4fd-81e5-4707-8180-d3553494c0a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fbaf6d894e64f19bf041e7be20a6242'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:33.254510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-86cbc4fd-81e5-4707-8180-d3553494c0a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fbaf6d894e64f19bf041e7be20a6242'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:33.255186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-86cbc4fd-81e5-4707-8180-d3553494c0a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fbaf6d894e64f19bf041e7be20a6242'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:33.263218 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-86cbc4fd-81e5-4707-8180-d3553494c0a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fbaf6d894e64f19bf041e7be20a6242'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:33.271860 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:33.279952 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-86cbc4fd-81e5-4707-8180-d3553494c0a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fbaf6d894e64f19bf041e7be20a6242'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:33.287648 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1378/2740] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:42:33 2022] DELETE /identity/v3/users/2fbaf6d894e64f19bf041e7be20a6242 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:33.471870 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7253c-e498-400d-87a3-740405c9e0ef service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:33.482462 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7253c-e498-400d-87a3-740405c9e0ef service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:33.581396 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e438b1-775b-48a8-b84d-cfa1de3cc845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:33.582297 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e438b1-775b-48a8-b84d-cfa1de3cc845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:33.582578 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e438b1-775b-48a8-b84d-cfa1de3cc845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:33.582854 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e438b1-775b-48a8-b84d-cfa1de3cc845 admin admin] PATH_INFO: `/v3/projects/34fb554e0cf14c7ab3a9f538273c289e` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:33.584213 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e438b1-775b-48a8-b84d-cfa1de3cc845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34fb554e0cf14c7ab3a9f538273c289e)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:33.586232 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e438b1-775b-48a8-b84d-cfa1de3cc845 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:33.633138 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:33.647454 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-31e438b1-775b-48a8-b84d-cfa1de3cc845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34fb554e0cf14c7ab3a9f538273c289e', 'request_id': 'req-31e438b1-775b-48a8-b84d-cfa1de3cc845'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:33.657645 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-31e438b1-775b-48a8-b84d-cfa1de3cc845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34fb554e0cf14c7ab3a9f538273c289e', 'request_id': 'req-31e438b1-775b-48a8-b84d-cfa1de3cc845'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:33.664618 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-31e438b1-775b-48a8-b84d-cfa1de3cc845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34fb554e0cf14c7ab3a9f538273c289e', 'request_id': 'req-31e438b1-775b-48a8-b84d-cfa1de3cc845'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:33.675816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-31e438b1-775b-48a8-b84d-cfa1de3cc845 admin admin] The token cache is being invalidate because project 34fb554e0cf14c7ab3a9f538273c289e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:42:33.676114 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-31e438b1-775b-48a8-b84d-cfa1de3cc845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:33.676758 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-31e438b1-775b-48a8-b84d-cfa1de3cc845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:33.678400 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1363/2741] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:42:33 2022] DELETE /identity/v3/projects/34fb554e0cf14c7ab3a9f538273c289e => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:33.683337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0c0db-9c2c-4d7e-855e-4f5c400d248b None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:33.683652 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0c0db-9c2c-4d7e-855e-4f5c400d248b None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:33.683921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0c0db-9c2c-4d7e-855e-4f5c400d248b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:33.704305 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-a7a0c0db-9c2c-4d7e-855e-4f5c400d248b None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:33.705618 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-a7a0c0db-9c2c-4d7e-855e-4f5c400d248b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:33.774935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1379/2742] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:33.784557 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9088b-664b-4d84-8623-93011d4b54ca None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:33.784846 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9088b-664b-4d84-8623-93011d4b54ca None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:33.785159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9088b-664b-4d84-8623-93011d4b54ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:33.805976 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-abc9088b-664b-4d84-8623-93011d4b54ca None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:33.807253 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-abc9088b-664b-4d84-8623-93011d4b54ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:33.818002 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1364/2743] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:33.823809 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0c0db-9c2c-4d7e-855e-4f5c400d248b None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:33.835603 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0c0db-9c2c-4d7e-855e-4f5c400d248b None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:33.840868 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9afed-356a-4c8a-90b2-76fcaf5b25f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:33.841591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9afed-356a-4c8a-90b2-76fcaf5b25f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:33.841888 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9afed-356a-4c8a-90b2-76fcaf5b25f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:33.842172 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9afed-356a-4c8a-90b2-76fcaf5b25f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:33.842929 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9afed-356a-4c8a-90b2-76fcaf5b25f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:42:33.843249 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9afed-356a-4c8a-90b2-76fcaf5b25f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:33.845071 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9afed-356a-4c8a-90b2-76fcaf5b25f9 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:33.862025 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1380/2744] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:42:33.874579 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9088b-664b-4d84-8623-93011d4b54ca None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:33.890434 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9088b-664b-4d84-8623-93011d4b54ca None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:33.895440 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e7f16-3286-479e-8fe8-a4ab8ba64451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:33.896122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e7f16-3286-479e-8fe8-a4ab8ba64451 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:33.896410 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e7f16-3286-479e-8fe8-a4ab8ba64451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:33.896722 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e7f16-3286-479e-8fe8-a4ab8ba64451 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:33.897463 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e7f16-3286-479e-8fe8-a4ab8ba64451 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:33.899371 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e7f16-3286-479e-8fe8-a4ab8ba64451 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:33.939213 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1365/2745] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:42:33 2022] POST /identity/v3/projects => generated 391 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:42:33.943539 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9afed-356a-4c8a-90b2-76fcaf5b25f9 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:33.953858 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9afed-356a-4c8a-90b2-76fcaf5b25f9 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:34.018365 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88f50a-e762-4c36-ba43-42f2995ebc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:34.019062 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88f50a-e762-4c36-ba43-42f2995ebc4d admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:34.019364 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88f50a-e762-4c36-ba43-42f2995ebc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:34.019647 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88f50a-e762-4c36-ba43-42f2995ebc4d admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:34.020420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88f50a-e762-4c36-ba43-42f2995ebc4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:34.022546 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88f50a-e762-4c36-ba43-42f2995ebc4d admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:34.048294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1381/2746] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:42:33 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:42:34.054339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e7f16-3286-479e-8fe8-a4ab8ba64451 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:34.071127 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e7f16-3286-479e-8fe8-a4ab8ba64451 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:34.079325 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e732de-851f-4213-9690-1a0acc481761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:34.080122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e732de-851f-4213-9690-1a0acc481761 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:34.080483 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e732de-851f-4213-9690-1a0acc481761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:34.080837 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e732de-851f-4213-9690-1a0acc481761 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:34.081701 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e732de-851f-4213-9690-1a0acc481761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:34.083387 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e732de-851f-4213-9690-1a0acc481761 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:34.103779 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1366/2747] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:42:34.108077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88f50a-e762-4c36-ba43-42f2995ebc4d admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:34.119621 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88f50a-e762-4c36-ba43-42f2995ebc4d admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:34.126006 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf3d28-2c7c-4e3c-be95-5b8621dd7836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:34.126311 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf3d28-2c7c-4e3c-be95-5b8621dd7836 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:34.126632 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf3d28-2c7c-4e3c-be95-5b8621dd7836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:34.126916 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf3d28-2c7c-4e3c-be95-5b8621dd7836 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:34.127663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf3d28-2c7c-4e3c-be95-5b8621dd7836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:34.129339 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf3d28-2c7c-4e3c-be95-5b8621dd7836 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:34.141696 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1382/2748] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:42:34.149250 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e732de-851f-4213-9690-1a0acc481761 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:34.162197 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e732de-851f-4213-9690-1a0acc481761 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:34.168687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97d663-9193-4f99-a225-7d98d4eef0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:34.169647 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97d663-9193-4f99-a225-7d98d4eef0ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:34.169932 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97d663-9193-4f99-a225-7d98d4eef0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:34.170214 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97d663-9193-4f99-a225-7d98d4eef0ab admin admin] PATH_INFO: `/v3/projects/77d6b3dffeca466f9e6f93e59cc881c5/users/41bc9880a90a4f5a90739b95b911b783/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:34.188667 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da97d663-9193-4f99-a225-7d98d4eef0ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d6b3dffeca466f9e6f93e59cc881c5, user_id=41bc9880a90a4f5a90739b95b911b783, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:34.191057 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da97d663-9193-4f99-a225-7d98d4eef0ab admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:34.208877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1367/2749] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:42:34 2022] PUT /identity/v3/projects/77d6b3dffeca466f9e6f93e59cc881c5/users/41bc9880a90a4f5a90739b95b911b783/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:34.218922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea68a8d-fcd9-460d-b1ff-32c13ea3b9d7 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:34.219602 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea68a8d-fcd9-460d-b1ff-32c13ea3b9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:34.219602 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea68a8d-fcd9-460d-b1ff-32c13ea3b9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:34.253816 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-bea68a8d-fcd9-460d-b1ff-32c13ea3b9d7 None None] MFA Rules not processed for user `41bc9880a90a4f5a90739b95b911b783`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:34.255108 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-bea68a8d-fcd9-460d-b1ff-32c13ea3b9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:34.323926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:34.350776 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1383/2750] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:42:34.355843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97d663-9193-4f99-a225-7d98d4eef0ab admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:34.432052 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97d663-9193-4f99-a225-7d98d4eef0ab admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:34.436891 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af536e56-6042-4183-bffe-c87957273e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:34.437640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af536e56-6042-4183-bffe-c87957273e13 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:34.437932 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af536e56-6042-4183-bffe-c87957273e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:34.438215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af536e56-6042-4183-bffe-c87957273e13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:34.439076 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af536e56-6042-4183-bffe-c87957273e13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:34.441139 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af536e56-6042-4183-bffe-c87957273e13 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:34.489259 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1368/2751] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:42:34 2022] POST /identity/v3/projects => generated 389 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:42:34.491839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea68a8d-fcd9-460d-b1ff-32c13ea3b9d7 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:34.512681 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea68a8d-fcd9-460d-b1ff-32c13ea3b9d7 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:34.577447 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eeacb0-0803-4786-9b71-a12a5e046699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:34.578800 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eeacb0-0803-4786-9b71-a12a5e046699 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:34.579092 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eeacb0-0803-4786-9b71-a12a5e046699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:34.580006 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eeacb0-0803-4786-9b71-a12a5e046699 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:34.580143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eeacb0-0803-4786-9b71-a12a5e046699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:34.582066 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eeacb0-0803-4786-9b71-a12a5e046699 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:34.603014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1384/2752] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:42:34 2022] POST /identity/v3/users => generated 301 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:42:34.607450 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af536e56-6042-4183-bffe-c87957273e13 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:34.618679 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af536e56-6042-4183-bffe-c87957273e13 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:34.623843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d88d4-e1dd-4b53-a7d8-f38e61c0a6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:34.624526 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d88d4-e1dd-4b53-a7d8-f38e61c0a6da admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:34.624848 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d88d4-e1dd-4b53-a7d8-f38e61c0a6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:34.625192 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d88d4-e1dd-4b53-a7d8-f38e61c0a6da admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:34.626135 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98d88d4-e1dd-4b53-a7d8-f38e61c0a6da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:34.627726 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98d88d4-e1dd-4b53-a7d8-f38e61c0a6da admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:34.642925 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1369/2753] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:42:34.644072 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eeacb0-0803-4786-9b71-a12a5e046699 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:34.654741 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eeacb0-0803-4786-9b71-a12a5e046699 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:34.660010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddde236-d438-4556-b0dd-60686673ff45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:34.660899 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddde236-d438-4556-b0dd-60686673ff45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:34.661236 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddde236-d438-4556-b0dd-60686673ff45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:34.661527 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddde236-d438-4556-b0dd-60686673ff45 admin admin] PATH_INFO: `/v3/projects/c722153453224bd3b70644681e46cd77/users/c189f13072f74dc389e22dd46eb72031/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:34.677779 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddde236-d438-4556-b0dd-60686673ff45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c722153453224bd3b70644681e46cd77, user_id=c189f13072f74dc389e22dd46eb72031, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:34.680442 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddde236-d438-4556-b0dd-60686673ff45 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:34.694682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1385/2754] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:42:34 2022] PUT /identity/v3/projects/c722153453224bd3b70644681e46cd77/users/c189f13072f74dc389e22dd46eb72031/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:34.698681 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d88d4-e1dd-4b53-a7d8-f38e61c0a6da admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:34.709745 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d88d4-e1dd-4b53-a7d8-f38e61c0a6da admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:34.740204 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:34.774610 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7aa52-151e-447c-af79-e40019ead1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:34.775403 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7aa52-151e-447c-af79-e40019ead1e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:34.775622 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7aa52-151e-447c-af79-e40019ead1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:34.775862 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7aa52-151e-447c-af79-e40019ead1e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:34.776605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7aa52-151e-447c-af79-e40019ead1e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:34.778262 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7aa52-151e-447c-af79-e40019ead1e6 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:34.789580 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1370/2755] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:42:34.793615 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddde236-d438-4556-b0dd-60686673ff45 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:34.806062 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddde236-d438-4556-b0dd-60686673ff45 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:34.811839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55eed4d-bd13-4574-94f1-baf4ea801042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:34.813842 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55eed4d-bd13-4574-94f1-baf4ea801042 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:34.814135 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55eed4d-bd13-4574-94f1-baf4ea801042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:34.814428 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55eed4d-bd13-4574-94f1-baf4ea801042 admin admin] PATH_INFO: `/v3/projects/c722153453224bd3b70644681e46cd77/users/c189f13072f74dc389e22dd46eb72031/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:34.818009 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55eed4d-bd13-4574-94f1-baf4ea801042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c722153453224bd3b70644681e46cd77, user_id=c189f13072f74dc389e22dd46eb72031, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:34.821266 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55eed4d-bd13-4574-94f1-baf4ea801042 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:34.833457 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1386/2756] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:42:34 2022] PUT /identity/v3/projects/c722153453224bd3b70644681e46cd77/users/c189f13072f74dc389e22dd46eb72031/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:34.844992 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc749b-3f60-4f82-ac9f-7130a15f83a8 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:34.845330 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc749b-3f60-4f82-ac9f-7130a15f83a8 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:34.845620 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc749b-3f60-4f82-ac9f-7130a15f83a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:34.877898 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-3cfc749b-3f60-4f82-ac9f-7130a15f83a8 None None] MFA Rules not processed for user `c189f13072f74dc389e22dd46eb72031`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:34.879054 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3cfc749b-3f60-4f82-ac9f-7130a15f83a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:34.956006 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1371/2757] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:34 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:34.966580 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55eed4d-bd13-4574-94f1-baf4ea801042 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:34.968299 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-a55eed4d-bd13-4574-94f1-baf4ea801042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:35.030541 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55eed4d-bd13-4574-94f1-baf4ea801042 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:35.034487 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06906594-ac10-4b68-b82e-3480a149b506 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:35.035147 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06906594-ac10-4b68-b82e-3480a149b506 service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:35.035439 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06906594-ac10-4b68-b82e-3480a149b506 service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:35.035723 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06906594-ac10-4b68-b82e-3480a149b506 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:35.045181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06906594-ac10-4b68-b82e-3480a149b506 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:35.046770 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06906594-ac10-4b68-b82e-3480a149b506 service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:35.057206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1387/2758] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:34 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:42:35.104594 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc749b-3f60-4f82-ac9f-7130a15f83a8 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:35.106053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3cfc749b-3f60-4f82-ac9f-7130a15f83a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:35.118620 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc749b-3f60-4f82-ac9f-7130a15f83a8 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:35.122440 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55656e7-53e4-45a8-8cd9-75c2abee8628 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:35.123172 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55656e7-53e4-45a8-8cd9-75c2abee8628 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:35.123552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55656e7-53e4-45a8-8cd9-75c2abee8628 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:35.123893 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55656e7-53e4-45a8-8cd9-75c2abee8628 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:35.195694 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55656e7-53e4-45a8-8cd9-75c2abee8628 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:35.197434 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55656e7-53e4-45a8-8cd9-75c2abee8628 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:35.208642 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1372/2759] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:42:38.571552 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06906594-ac10-4b68-b82e-3480a149b506 service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:38.616460 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:38.638589 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06906594-ac10-4b68-b82e-3480a149b506 service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:38.644475 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1a5ec-39fb-4e11-9244-4ea33fd5b7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:38.645422 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1a5ec-39fb-4e11-9244-4ea33fd5b7f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:38.645718 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1a5ec-39fb-4e11-9244-4ea33fd5b7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:38.646002 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1a5ec-39fb-4e11-9244-4ea33fd5b7f3 admin admin] PATH_INFO: `/v3/users/41bc9880a90a4f5a90739b95b911b783` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:38.648748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1a5ec-39fb-4e11-9244-4ea33fd5b7f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41bc9880a90a4f5a90739b95b911b783)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:38.652796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1a5ec-39fb-4e11-9244-4ea33fd5b7f3 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:38.719662 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-51b1a5ec-39fb-4e11-9244-4ea33fd5b7f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41bc9880a90a4f5a90739b95b911b783'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:38.727135 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-51b1a5ec-39fb-4e11-9244-4ea33fd5b7f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41bc9880a90a4f5a90739b95b911b783'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:38.728433 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-51b1a5ec-39fb-4e11-9244-4ea33fd5b7f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41bc9880a90a4f5a90739b95b911b783'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:38.738738 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-51b1a5ec-39fb-4e11-9244-4ea33fd5b7f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41bc9880a90a4f5a90739b95b911b783'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:38.754688 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-51b1a5ec-39fb-4e11-9244-4ea33fd5b7f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41bc9880a90a4f5a90739b95b911b783'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:38.760930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1388/2760] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:42:38 2022] DELETE /identity/v3/users/41bc9880a90a4f5a90739b95b911b783 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:38.766139 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4087b-158c-4732-8787-ec75a510e4b0 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:38.766771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4087b-158c-4732-8787-ec75a510e4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:38.766771 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4087b-158c-4732-8787-ec75a510e4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:38.784352 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-19b4087b-158c-4732-8787-ec75a510e4b0 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:38.785618 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-19b4087b-158c-4732-8787-ec75a510e4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:38.835225 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:38.850793 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1373/2761] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:42:38.861730 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1a5ec-39fb-4e11-9244-4ea33fd5b7f3 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:38.863148 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-51b1a5ec-39fb-4e11-9244-4ea33fd5b7f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:38.920807 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1a5ec-39fb-4e11-9244-4ea33fd5b7f3 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:38.925336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2dd4d-2e68-4c81-af59-fea1a0fc3833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:38.925991 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2dd4d-2e68-4c81-af59-fea1a0fc3833 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:38.926277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2dd4d-2e68-4c81-af59-fea1a0fc3833 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:38.926556 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2dd4d-2e68-4c81-af59-fea1a0fc3833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:38.936446 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2dd4d-2e68-4c81-af59-fea1a0fc3833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:38.938092 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2dd4d-2e68-4c81-af59-fea1a0fc3833 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:38.947832 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1389/2762] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:42:39.252880 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4087b-158c-4732-8787-ec75a510e4b0 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:39.255110 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-19b4087b-158c-4732-8787-ec75a510e4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:39.269409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4087b-158c-4732-8787-ec75a510e4b0 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:39.274394 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c33fb2-e3a0-4771-851b-8c0d52281478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:39.275408 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c33fb2-e3a0-4771-851b-8c0d52281478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:39.275776 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c33fb2-e3a0-4771-851b-8c0d52281478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:39.276094 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c33fb2-e3a0-4771-851b-8c0d52281478 admin admin] PATH_INFO: `/v3/projects/77d6b3dffeca466f9e6f93e59cc881c5` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:39.277424 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c33fb2-e3a0-4771-851b-8c0d52281478 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77d6b3dffeca466f9e6f93e59cc881c5)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:39.279676 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c33fb2-e3a0-4771-851b-8c0d52281478 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:39.339640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b5c33fb2-e3a0-4771-851b-8c0d52281478 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77d6b3dffeca466f9e6f93e59cc881c5', 'request_id': 'req-b5c33fb2-e3a0-4771-851b-8c0d52281478'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:39.354036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b5c33fb2-e3a0-4771-851b-8c0d52281478 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77d6b3dffeca466f9e6f93e59cc881c5', 'request_id': 'req-b5c33fb2-e3a0-4771-851b-8c0d52281478'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:39.361602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b5c33fb2-e3a0-4771-851b-8c0d52281478 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77d6b3dffeca466f9e6f93e59cc881c5', 'request_id': 'req-b5c33fb2-e3a0-4771-851b-8c0d52281478'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:39.373520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b5c33fb2-e3a0-4771-851b-8c0d52281478 admin admin] The token cache is being invalidate because project 77d6b3dffeca466f9e6f93e59cc881c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:42:39.374054 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b5c33fb2-e3a0-4771-851b-8c0d52281478 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:39.376041 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-b5c33fb2-e3a0-4771-851b-8c0d52281478 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:39.377957 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1374/2763] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:42:39 2022] DELETE /identity/v3/projects/77d6b3dffeca466f9e6f93e59cc881c5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:39.381948 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2dd4d-2e68-4c81-af59-fea1a0fc3833 service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:39.392382 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2dd4d-2e68-4c81-af59-fea1a0fc3833 service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:39.450653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21579a3c-e857-499b-a652-38cce76b3e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:39.451682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21579a3c-e857-499b-a652-38cce76b3e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:39.452012 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21579a3c-e857-499b-a652-38cce76b3e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:39.452443 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21579a3c-e857-499b-a652-38cce76b3e29 admin admin] PATH_INFO: `/v3/users/c189f13072f74dc389e22dd46eb72031` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:39.454885 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21579a3c-e857-499b-a652-38cce76b3e29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c189f13072f74dc389e22dd46eb72031)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:39.457101 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21579a3c-e857-499b-a652-38cce76b3e29 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:39.459548 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:39.506736 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-21579a3c-e857-499b-a652-38cce76b3e29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c189f13072f74dc389e22dd46eb72031'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:39.514585 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-21579a3c-e857-499b-a652-38cce76b3e29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c189f13072f74dc389e22dd46eb72031'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:39.516034 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-21579a3c-e857-499b-a652-38cce76b3e29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c189f13072f74dc389e22dd46eb72031'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:39.523307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-21579a3c-e857-499b-a652-38cce76b3e29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c189f13072f74dc389e22dd46eb72031'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:39.535950 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-21579a3c-e857-499b-a652-38cce76b3e29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c189f13072f74dc389e22dd46eb72031'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:39.542123 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1390/2764] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:42:39 2022] DELETE /identity/v3/users/c189f13072f74dc389e22dd46eb72031 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:39.821050 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c33fb2-e3a0-4771-851b-8c0d52281478 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:39.822693 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b5c33fb2-e3a0-4771-851b-8c0d52281478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:39.836832 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c33fb2-e3a0-4771-851b-8c0d52281478 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:39.897514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b03074-dfa4-466e-9849-54167f6fd8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:39.898318 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b03074-dfa4-466e-9849-54167f6fd8d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:39.898606 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b03074-dfa4-466e-9849-54167f6fd8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:39.898888 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b03074-dfa4-466e-9849-54167f6fd8d0 admin admin] PATH_INFO: `/v3/projects/c722153453224bd3b70644681e46cd77` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:39.900095 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b03074-dfa4-466e-9849-54167f6fd8d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c722153453224bd3b70644681e46cd77)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:39.902098 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b03074-dfa4-466e-9849-54167f6fd8d0 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:39.956168 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e1b03074-dfa4-466e-9849-54167f6fd8d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c722153453224bd3b70644681e46cd77', 'request_id': 'req-e1b03074-dfa4-466e-9849-54167f6fd8d0'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:39.967573 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e1b03074-dfa4-466e-9849-54167f6fd8d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c722153453224bd3b70644681e46cd77', 'request_id': 'req-e1b03074-dfa4-466e-9849-54167f6fd8d0'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:39.974904 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e1b03074-dfa4-466e-9849-54167f6fd8d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c722153453224bd3b70644681e46cd77', 'request_id': 'req-e1b03074-dfa4-466e-9849-54167f6fd8d0'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:39.987226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e1b03074-dfa4-466e-9849-54167f6fd8d0 admin admin] The token cache is being invalidate because project c722153453224bd3b70644681e46cd77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:42:39.987524 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e1b03074-dfa4-466e-9849-54167f6fd8d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:39.988189 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e1b03074-dfa4-466e-9849-54167f6fd8d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:39.989816 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1375/2765] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:42:39 2022] DELETE /identity/v3/projects/c722153453224bd3b70644681e46cd77 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:39.994755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713366b5-d57f-4111-9545-45726e8ffec4 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:39.995257 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713366b5-d57f-4111-9545-45726e8ffec4 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:39.995407 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713366b5-d57f-4111-9545-45726e8ffec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:40.013855 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-713366b5-d57f-4111-9545-45726e8ffec4 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:40.015620 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-713366b5-d57f-4111-9545-45726e8ffec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:40.076923 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1391/2766] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:40.085914 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd07fc6-38e6-40ab-943d-fbbc0b4e356b None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:40.086209 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd07fc6-38e6-40ab-943d-fbbc0b4e356b None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:40.086496 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd07fc6-38e6-40ab-943d-fbbc0b4e356b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:40.097216 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:40.110258 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-1cd07fc6-38e6-40ab-943d-fbbc0b4e356b None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:40.111514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-1cd07fc6-38e6-40ab-943d-fbbc0b4e356b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:40.119531 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1376/2767] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:40.125522 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713366b5-d57f-4111-9545-45726e8ffec4 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:40.135131 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713366b5-d57f-4111-9545-45726e8ffec4 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:40.139670 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd376ec-c571-4956-8a09-bbf54f4dae99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:40.141043 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd376ec-c571-4956-8a09-bbf54f4dae99 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:40.141389 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd376ec-c571-4956-8a09-bbf54f4dae99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:40.141673 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd376ec-c571-4956-8a09-bbf54f4dae99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:40.142408 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd376ec-c571-4956-8a09-bbf54f4dae99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80152) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:42:40.142747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd376ec-c571-4956-8a09-bbf54f4dae99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:40.144342 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd376ec-c571-4956-8a09-bbf54f4dae99 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:40.156932 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1392/2768] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:42:40.161365 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd07fc6-38e6-40ab-943d-fbbc0b4e356b None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:40.170490 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd07fc6-38e6-40ab-943d-fbbc0b4e356b None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:40.176153 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec1187-5602-4cae-a01f-a182311f9938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:40.176815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec1187-5602-4cae-a01f-a182311f9938 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:40.177177 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec1187-5602-4cae-a01f-a182311f9938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:40.177393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec1187-5602-4cae-a01f-a182311f9938 admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:40.178102 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec1187-5602-4cae-a01f-a182311f9938 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:40.179976 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec1187-5602-4cae-a01f-a182311f9938 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:40.207504 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1377/2769] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:42:40 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:42:40.211723 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd376ec-c571-4956-8a09-bbf54f4dae99 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:40.221714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd376ec-c571-4956-8a09-bbf54f4dae99 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:40.278730 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47173a5b-b30c-4823-bebd-c7bea1eccc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:40.279400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47173a5b-b30c-4823-bebd-c7bea1eccc60 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:40.279690 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47173a5b-b30c-4823-bebd-c7bea1eccc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:40.279971 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47173a5b-b30c-4823-bebd-c7bea1eccc60 admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:40.280774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47173a5b-b30c-4823-bebd-c7bea1eccc60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:40.282765 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47173a5b-b30c-4823-bebd-c7bea1eccc60 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:40.297546 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:40.304299 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1393/2770] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:42:40 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:42:40.308403 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec1187-5602-4cae-a01f-a182311f9938 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:40.319539 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec1187-5602-4cae-a01f-a182311f9938 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:40.324284 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49988e-0a32-49b4-a1f8-9b0242fffda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:40.324987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49988e-0a32-49b4-a1f8-9b0242fffda9 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:40.325317 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49988e-0a32-49b4-a1f8-9b0242fffda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:40.325596 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49988e-0a32-49b4-a1f8-9b0242fffda9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:40.326336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c49988e-0a32-49b4-a1f8-9b0242fffda9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:40.328590 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c49988e-0a32-49b4-a1f8-9b0242fffda9 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:40.339569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1378/2771] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:42:40.344180 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47173a5b-b30c-4823-bebd-c7bea1eccc60 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:40.354504 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47173a5b-b30c-4823-bebd-c7bea1eccc60 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:40.359236 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4250af-e754-4b10-a785-34db60422156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:40.359917 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4250af-e754-4b10-a785-34db60422156 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:40.360211 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4250af-e754-4b10-a785-34db60422156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:40.360496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4250af-e754-4b10-a785-34db60422156 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:40.361266 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4250af-e754-4b10-a785-34db60422156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:40.363016 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4250af-e754-4b10-a785-34db60422156 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:40.373225 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1394/2772] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:42:40.379163 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49988e-0a32-49b4-a1f8-9b0242fffda9 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:40.389406 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49988e-0a32-49b4-a1f8-9b0242fffda9 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:40.395860 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce868308-9dfa-4a40-80a6-50cf684398f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:40.396777 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce868308-9dfa-4a40-80a6-50cf684398f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:40.397095 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce868308-9dfa-4a40-80a6-50cf684398f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:40.397424 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce868308-9dfa-4a40-80a6-50cf684398f1 admin admin] PATH_INFO: `/v3/projects/40840773d04e4412bdcbebbc995eb8bd/users/96dc884a9cdc4dc795d5faada1a18999/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:40.415822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce868308-9dfa-4a40-80a6-50cf684398f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40840773d04e4412bdcbebbc995eb8bd, user_id=96dc884a9cdc4dc795d5faada1a18999, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:40.418876 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce868308-9dfa-4a40-80a6-50cf684398f1 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:40.432871 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1379/2773] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:42:40 2022] PUT /identity/v3/projects/40840773d04e4412bdcbebbc995eb8bd/users/96dc884a9cdc4dc795d5faada1a18999/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:40.444247 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb26e91-0635-4b57-b90f-f3fd6dc38ab3 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:40.444568 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb26e91-0635-4b57-b90f-f3fd6dc38ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:40.444857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb26e91-0635-4b57-b90f-f3fd6dc38ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:40.473968 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-0bb26e91-0635-4b57-b90f-f3fd6dc38ab3 None None] MFA Rules not processed for user `96dc884a9cdc4dc795d5faada1a18999`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:40.475209 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-0bb26e91-0635-4b57-b90f-f3fd6dc38ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:40.550424 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1395/2774] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:40.554959 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce868308-9dfa-4a40-80a6-50cf684398f1 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:40.626027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce868308-9dfa-4a40-80a6-50cf684398f1 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:40.630955 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63cc84-0d51-4f4c-bf15-18ad1fa5131d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:40.631704 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63cc84-0d51-4f4c-bf15-18ad1fa5131d admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:40.631999 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63cc84-0d51-4f4c-bf15-18ad1fa5131d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:40.632282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63cc84-0d51-4f4c-bf15-18ad1fa5131d admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:40.633008 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63cc84-0d51-4f4c-bf15-18ad1fa5131d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:40.635021 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63cc84-0d51-4f4c-bf15-18ad1fa5131d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:40.667572 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1380/2775] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:42:40 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:42:40.669090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb26e91-0635-4b57-b90f-f3fd6dc38ab3 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:40.685545 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb26e91-0635-4b57-b90f-f3fd6dc38ab3 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:40.767215 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f8afb0-e62e-4355-9874-acea24195e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:40.767487 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f8afb0-e62e-4355-9874-acea24195e8a admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:40.767758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f8afb0-e62e-4355-9874-acea24195e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:40.768048 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f8afb0-e62e-4355-9874-acea24195e8a admin admin] PATH_INFO: `/v3/users` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:40.768777 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f8afb0-e62e-4355-9874-acea24195e8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:40.771097 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f8afb0-e62e-4355-9874-acea24195e8a admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:40.794108 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1396/2776] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:42:40 2022] POST /identity/v3/users => generated 298 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:42:40.799023 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63cc84-0d51-4f4c-bf15-18ad1fa5131d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:40.810062 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63cc84-0d51-4f4c-bf15-18ad1fa5131d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:40.816351 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8cbda9-fe76-4c88-bc5d-5eb0dafbddc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:40.817185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cbda9-fe76-4c88-bc5d-5eb0dafbddc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:40.817581 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cbda9-fe76-4c88-bc5d-5eb0dafbddc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:40.817964 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cbda9-fe76-4c88-bc5d-5eb0dafbddc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:40.818783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8cbda9-fe76-4c88-bc5d-5eb0dafbddc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:40.820456 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8cbda9-fe76-4c88-bc5d-5eb0dafbddc6 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:40.835277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1381/2777] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:42:40.836419 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f8afb0-e62e-4355-9874-acea24195e8a admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:40.847033 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f8afb0-e62e-4355-9874-acea24195e8a admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:40.851672 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e021df4-800e-46c8-82ed-00c53f3bae5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:40.852563 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e021df4-800e-46c8-82ed-00c53f3bae5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:40.852847 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e021df4-800e-46c8-82ed-00c53f3bae5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:40.853130 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e021df4-800e-46c8-82ed-00c53f3bae5f admin admin] PATH_INFO: `/v3/projects/200392e7b5584426ace8eb0068e78681/users/e14238ed47474a4fbcaa5f79e632b946/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:40.868922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e021df4-800e-46c8-82ed-00c53f3bae5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=200392e7b5584426ace8eb0068e78681, user_id=e14238ed47474a4fbcaa5f79e632b946, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:40.871587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e021df4-800e-46c8-82ed-00c53f3bae5f admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:40.885137 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1397/2778] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:42:40 2022] PUT /identity/v3/projects/200392e7b5584426ace8eb0068e78681/users/e14238ed47474a4fbcaa5f79e632b946/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:40.889131 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8cbda9-fe76-4c88-bc5d-5eb0dafbddc6 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:40.900068 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8cbda9-fe76-4c88-bc5d-5eb0dafbddc6 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:40.958974 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4fed7-06fc-4672-8e47-32780c1b0084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:40.959576 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4fed7-06fc-4672-8e47-32780c1b0084 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:40.959868 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4fed7-06fc-4672-8e47-32780c1b0084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:40.960156 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4fed7-06fc-4672-8e47-32780c1b0084 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:40.960969 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4fed7-06fc-4672-8e47-32780c1b0084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:40.962603 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4fed7-06fc-4672-8e47-32780c1b0084 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:40.973008 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1382/2779] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:42:40.977223 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e021df4-800e-46c8-82ed-00c53f3bae5f admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:40.988554 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e021df4-800e-46c8-82ed-00c53f3bae5f admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:40.993459 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361d1f8-d512-463e-b20c-9a7eeaa00e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:40.994405 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361d1f8-d512-463e-b20c-9a7eeaa00e3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:40.994698 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361d1f8-d512-463e-b20c-9a7eeaa00e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:40.994985 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361d1f8-d512-463e-b20c-9a7eeaa00e3f admin admin] PATH_INFO: `/v3/projects/200392e7b5584426ace8eb0068e78681/users/e14238ed47474a4fbcaa5f79e632b946/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:40.997102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c361d1f8-d512-463e-b20c-9a7eeaa00e3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=200392e7b5584426ace8eb0068e78681, user_id=e14238ed47474a4fbcaa5f79e632b946, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:41.000714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c361d1f8-d512-463e-b20c-9a7eeaa00e3f admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:41.011588 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1398/2780] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:42:40 2022] PUT /identity/v3/projects/200392e7b5584426ace8eb0068e78681/users/e14238ed47474a4fbcaa5f79e632b946/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:41.025328 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f874a28-1217-4888-b327-2eba3605d40d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:41.025328 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f874a28-1217-4888-b327-2eba3605d40d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:41.025734 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f874a28-1217-4888-b327-2eba3605d40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:41.045648 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:41.057541 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-6f874a28-1217-4888-b327-2eba3605d40d None None] MFA Rules not processed for user `e14238ed47474a4fbcaa5f79e632b946`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:41.058789 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6f874a28-1217-4888-b327-2eba3605d40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:41.160575 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1383/2781] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:41.171888 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361d1f8-d512-463e-b20c-9a7eeaa00e3f admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:41.174158 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-c361d1f8-d512-463e-b20c-9a7eeaa00e3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:41.209139 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:41.241686 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361d1f8-d512-463e-b20c-9a7eeaa00e3f admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:41.245804 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eaf276-6928-4f19-bb9f-15a518e01739 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:41.246519 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eaf276-6928-4f19-bb9f-15a518e01739 service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:41.246691 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eaf276-6928-4f19-bb9f-15a518e01739 service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:41.246974 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eaf276-6928-4f19-bb9f-15a518e01739 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:41.257372 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eaf276-6928-4f19-bb9f-15a518e01739 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:41.258819 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eaf276-6928-4f19-bb9f-15a518e01739 service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:41.268764 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1399/2782] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:42:41.369686 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f874a28-1217-4888-b327-2eba3605d40d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:41.380209 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f874a28-1217-4888-b327-2eba3605d40d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:41.383819 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7f073-937c-48e8-9f45-d8ae7bded243 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:41.384534 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7f073-937c-48e8-9f45-d8ae7bded243 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:41.384825 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7f073-937c-48e8-9f45-d8ae7bded243 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:41.385112 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7f073-937c-48e8-9f45-d8ae7bded243 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:41.449109 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7f073-937c-48e8-9f45-d8ae7bded243 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:41.450861 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7f073-937c-48e8-9f45-d8ae7bded243 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:41.461763 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1384/2783] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:42:43.762643 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eaf276-6928-4f19-bb9f-15a518e01739 service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:43.764112 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-25eaf276-6928-4f19-bb9f-15a518e01739 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:43.833375 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eaf276-6928-4f19-bb9f-15a518e01739 service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:43.837316 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18be299-c7b2-412a-97bf-ef80fcc55df2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:43.838055 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18be299-c7b2-412a-97bf-ef80fcc55df2 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:43.838347 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18be299-c7b2-412a-97bf-ef80fcc55df2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:43.838662 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18be299-c7b2-412a-97bf-ef80fcc55df2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:43.840480 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-d18be299-c7b2-412a-97bf-ef80fcc55df2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:43.907241 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18be299-c7b2-412a-97bf-ef80fcc55df2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:43.908686 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18be299-c7b2-412a-97bf-ef80fcc55df2 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:43.918656 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1400/2784] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:43 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:42:44.056092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7f073-937c-48e8-9f45-d8ae7bded243 service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:44.135552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7f073-937c-48e8-9f45-d8ae7bded243 service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:44.141357 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9039953-4217-425b-8e8f-7e363ce66ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:44.142295 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9039953-4217-425b-8e8f-7e363ce66ae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:44.142641 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9039953-4217-425b-8e8f-7e363ce66ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:44.142931 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9039953-4217-425b-8e8f-7e363ce66ae1 admin admin] PATH_INFO: `/v3/users/96dc884a9cdc4dc795d5faada1a18999` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:44.144388 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9039953-4217-425b-8e8f-7e363ce66ae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96dc884a9cdc4dc795d5faada1a18999)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:44.146502 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9039953-4217-425b-8e8f-7e363ce66ae1 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:44.204316 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:44.217413 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e9039953-4217-425b-8e8f-7e363ce66ae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96dc884a9cdc4dc795d5faada1a18999'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:44.254614 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e9039953-4217-425b-8e8f-7e363ce66ae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96dc884a9cdc4dc795d5faada1a18999'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:44.255484 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e9039953-4217-425b-8e8f-7e363ce66ae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96dc884a9cdc4dc795d5faada1a18999'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:44.264755 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e9039953-4217-425b-8e8f-7e363ce66ae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96dc884a9cdc4dc795d5faada1a18999'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:44.278590 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e9039953-4217-425b-8e8f-7e363ce66ae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96dc884a9cdc4dc795d5faada1a18999'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:44.288115 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1385/2785] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:42:44 2022] DELETE /identity/v3/users/96dc884a9cdc4dc795d5faada1a18999 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:44.293202 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b2874-d28b-4fe2-bc28-aa279e77854d None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:44.293489 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b2874-d28b-4fe2-bc28-aa279e77854d None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:44.293816 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b2874-d28b-4fe2-bc28-aa279e77854d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:44.321613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-8c7b2874-d28b-4fe2-bc28-aa279e77854d None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:44.322839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-8c7b2874-d28b-4fe2-bc28-aa279e77854d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:44.396960 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1401/2786] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:44.404671 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9039953-4217-425b-8e8f-7e363ce66ae1 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:44.407348 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e9039953-4217-425b-8e8f-7e363ce66ae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:44.476844 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9039953-4217-425b-8e8f-7e363ce66ae1 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:44.481378 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f5feb-652a-4e7c-8130-a2649de0b4b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:44.482042 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f5feb-652a-4e7c-8130-a2649de0b4b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:44.482326 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f5feb-652a-4e7c-8130-a2649de0b4b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:44.482643 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f5feb-652a-4e7c-8130-a2649de0b4b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:44.494109 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f5feb-652a-4e7c-8130-a2649de0b4b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:44.495598 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f5feb-652a-4e7c-8130-a2649de0b4b4 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:44.506678 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1386/2787] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:42:44.845577 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b2874-d28b-4fe2-bc28-aa279e77854d None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:44.847333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-8c7b2874-d28b-4fe2-bc28-aa279e77854d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:44.866575 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b2874-d28b-4fe2-bc28-aa279e77854d None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:44.875422 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72a422-6238-4ae3-9765-ca97f1da4faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:44.876440 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72a422-6238-4ae3-9765-ca97f1da4faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:44.876740 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72a422-6238-4ae3-9765-ca97f1da4faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:44.877027 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72a422-6238-4ae3-9765-ca97f1da4faa admin admin] PATH_INFO: `/v3/projects/40840773d04e4412bdcbebbc995eb8bd` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:44.878339 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72a422-6238-4ae3-9765-ca97f1da4faa admin admin] RBAC: Authorizing `identity:delete_project(project_id=40840773d04e4412bdcbebbc995eb8bd)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:44.880270 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72a422-6238-4ae3-9765-ca97f1da4faa admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:44.898930 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:44.934420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-eb72a422-6238-4ae3-9765-ca97f1da4faa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40840773d04e4412bdcbebbc995eb8bd', 'request_id': 'req-eb72a422-6238-4ae3-9765-ca97f1da4faa'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:44.942413 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-eb72a422-6238-4ae3-9765-ca97f1da4faa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40840773d04e4412bdcbebbc995eb8bd', 'request_id': 'req-eb72a422-6238-4ae3-9765-ca97f1da4faa'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:44.950559 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-eb72a422-6238-4ae3-9765-ca97f1da4faa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40840773d04e4412bdcbebbc995eb8bd', 'request_id': 'req-eb72a422-6238-4ae3-9765-ca97f1da4faa'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:44.964990 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-eb72a422-6238-4ae3-9765-ca97f1da4faa admin admin] The token cache is being invalidate because project 40840773d04e4412bdcbebbc995eb8bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:42:44.965328 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-eb72a422-6238-4ae3-9765-ca97f1da4faa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:44.966005 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-eb72a422-6238-4ae3-9765-ca97f1da4faa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:44.967631 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1402/2788] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:42:44 2022] DELETE /identity/v3/projects/40840773d04e4412bdcbebbc995eb8bd => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:44.972464 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f5feb-652a-4e7c-8130-a2649de0b4b4 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:44.985082 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f5feb-652a-4e7c-8130-a2649de0b4b4 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:45.055277 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e6a7f-409e-44dc-9326-62575662044d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:45.056105 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e6a7f-409e-44dc-9326-62575662044d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:45.057098 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e6a7f-409e-44dc-9326-62575662044d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:45.057098 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e6a7f-409e-44dc-9326-62575662044d admin admin] PATH_INFO: `/v3/users/e14238ed47474a4fbcaa5f79e632b946` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:45.058666 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e6a7f-409e-44dc-9326-62575662044d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e14238ed47474a4fbcaa5f79e632b946)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:45.061132 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e6a7f-409e-44dc-9326-62575662044d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:45.121127 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4e6e6a7f-409e-44dc-9326-62575662044d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e14238ed47474a4fbcaa5f79e632b946'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:45.131452 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:45.137587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4e6e6a7f-409e-44dc-9326-62575662044d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e14238ed47474a4fbcaa5f79e632b946'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:45.138375 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4e6e6a7f-409e-44dc-9326-62575662044d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e14238ed47474a4fbcaa5f79e632b946'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:45.151486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4e6e6a7f-409e-44dc-9326-62575662044d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e14238ed47474a4fbcaa5f79e632b946'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:45.161718 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-4e6e6a7f-409e-44dc-9326-62575662044d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e14238ed47474a4fbcaa5f79e632b946'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:45.168367 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1387/2789] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:42:44 2022] DELETE /identity/v3/users/e14238ed47474a4fbcaa5f79e632b946 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:45.499366 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72a422-6238-4ae3-9765-ca97f1da4faa admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:45.501651 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-eb72a422-6238-4ae3-9765-ca97f1da4faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:45.529609 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72a422-6238-4ae3-9765-ca97f1da4faa admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:45.600868 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f86273-0438-4886-a503-4fcd67d80ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:45.601775 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f86273-0438-4886-a503-4fcd67d80ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:45.602058 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f86273-0438-4886-a503-4fcd67d80ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:45.602340 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f86273-0438-4886-a503-4fcd67d80ab2 admin admin] PATH_INFO: `/v3/projects/200392e7b5584426ace8eb0068e78681` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:45.604061 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f86273-0438-4886-a503-4fcd67d80ab2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=200392e7b5584426ace8eb0068e78681)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:45.606084 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f86273-0438-4886-a503-4fcd67d80ab2 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:45.660749 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d4f86273-0438-4886-a503-4fcd67d80ab2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '200392e7b5584426ace8eb0068e78681', 'request_id': 'req-d4f86273-0438-4886-a503-4fcd67d80ab2'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:45.666758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d4f86273-0438-4886-a503-4fcd67d80ab2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '200392e7b5584426ace8eb0068e78681', 'request_id': 'req-d4f86273-0438-4886-a503-4fcd67d80ab2'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:45.673630 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d4f86273-0438-4886-a503-4fcd67d80ab2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '200392e7b5584426ace8eb0068e78681', 'request_id': 'req-d4f86273-0438-4886-a503-4fcd67d80ab2'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:45.689922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d4f86273-0438-4886-a503-4fcd67d80ab2 admin admin] The token cache is being invalidate because project 200392e7b5584426ace8eb0068e78681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:42:45.689922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d4f86273-0438-4886-a503-4fcd67d80ab2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:45.690272 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d4f86273-0438-4886-a503-4fcd67d80ab2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:45.692136 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1403/2790] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:42:45 2022] DELETE /identity/v3/projects/200392e7b5584426ace8eb0068e78681 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:45.696921 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11c3f6-7b0e-463c-b03b-b5bb5ed8399d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:45.697252 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11c3f6-7b0e-463c-b03b-b5bb5ed8399d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:45.697544 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11c3f6-7b0e-463c-b03b-b5bb5ed8399d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:45.717857 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-7f11c3f6-7b0e-463c-b03b-b5bb5ed8399d None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:45.719356 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7f11c3f6-7b0e-463c-b03b-b5bb5ed8399d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:45.802083 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1388/2791] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:45.818354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d7ecb-2fc2-4ffc-a161-5071819cfec8 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:45.819111 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d7ecb-2fc2-4ffc-a161-5071819cfec8 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:45.819506 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d7ecb-2fc2-4ffc-a161-5071819cfec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:45.847002 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-df7d7ecb-2fc2-4ffc-a161-5071819cfec8 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:45.849121 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-df7d7ecb-2fc2-4ffc-a161-5071819cfec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:45.862075 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1404/2792] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:45.866397 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11c3f6-7b0e-463c-b03b-b5bb5ed8399d None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:45.876990 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11c3f6-7b0e-463c-b03b-b5bb5ed8399d None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:45.882367 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311851f-5694-4f93-968b-d7b5d59a05b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:45.883425 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311851f-5694-4f93-968b-d7b5d59a05b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:45.883913 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311851f-5694-4f93-968b-d7b5d59a05b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:45.884371 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311851f-5694-4f93-968b-d7b5d59a05b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:45.885693 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311851f-5694-4f93-968b-d7b5d59a05b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:42:45.886216 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311851f-5694-4f93-968b-d7b5d59a05b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:45.889266 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311851f-5694-4f93-968b-d7b5d59a05b0 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:45.905160 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1389/2793] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:42:45.910622 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d7ecb-2fc2-4ffc-a161-5071819cfec8 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:45.920336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d7ecb-2fc2-4ffc-a161-5071819cfec8 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:45.934371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e68e1-741f-495e-a46c-5c006955d7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:45.935069 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e68e1-741f-495e-a46c-5c006955d7e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:45.935370 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e68e1-741f-495e-a46c-5c006955d7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:45.935654 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e68e1-741f-495e-a46c-5c006955d7e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:45.936381 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e68e1-741f-495e-a46c-5c006955d7e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:45.939371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e68e1-741f-495e-a46c-5c006955d7e9 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:45.982480 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1405/2794] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:42:45 2022] POST /identity/v3/projects => generated 381 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:42:45.985010 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311851f-5694-4f93-968b-d7b5d59a05b0 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:45.997336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311851f-5694-4f93-968b-d7b5d59a05b0 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:46.078036 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd9736-8fc7-4c72-8396-830c18882ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:46.080399 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd9736-8fc7-4c72-8396-830c18882ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:46.081052 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd9736-8fc7-4c72-8396-830c18882ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:46.081833 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd9736-8fc7-4c72-8396-830c18882ef4 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:46.082924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd9736-8fc7-4c72-8396-830c18882ef4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:46.085117 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd9736-8fc7-4c72-8396-830c18882ef4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:46.110602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1390/2795] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:42:45 2022] POST /identity/v3/users => generated 291 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:42:46.114790 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e68e1-741f-495e-a46c-5c006955d7e9 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:46.127926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e68e1-741f-495e-a46c-5c006955d7e9 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:46.136638 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408af46-b9a5-4c20-8e56-49ba4aa556aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:46.138143 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408af46-b9a5-4c20-8e56-49ba4aa556aa admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:46.138653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408af46-b9a5-4c20-8e56-49ba4aa556aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:46.139778 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408af46-b9a5-4c20-8e56-49ba4aa556aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:46.142423 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408af46-b9a5-4c20-8e56-49ba4aa556aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:46.145711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408af46-b9a5-4c20-8e56-49ba4aa556aa admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:46.149509 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:46.173315 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1406/2796] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:42:46.173605 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd9736-8fc7-4c72-8396-830c18882ef4 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:46.186200 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd9736-8fc7-4c72-8396-830c18882ef4 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:46.192519 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e883702-bfa1-4ee5-a772-de20b90cf540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:46.193248 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e883702-bfa1-4ee5-a772-de20b90cf540 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:46.193538 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e883702-bfa1-4ee5-a772-de20b90cf540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:46.193822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e883702-bfa1-4ee5-a772-de20b90cf540 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:46.194563 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e883702-bfa1-4ee5-a772-de20b90cf540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:46.196178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e883702-bfa1-4ee5-a772-de20b90cf540 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:46.210040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1391/2797] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:42:46.214262 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408af46-b9a5-4c20-8e56-49ba4aa556aa admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:46.225216 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408af46-b9a5-4c20-8e56-49ba4aa556aa admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:46.230289 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d6172-9525-4985-b1a9-bfa50573777b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:46.231206 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d6172-9525-4985-b1a9-bfa50573777b admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:46.231494 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d6172-9525-4985-b1a9-bfa50573777b admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:46.231926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d6172-9525-4985-b1a9-bfa50573777b admin admin] PATH_INFO: `/v3/projects/e01be68d7d444f87bb454e140b04c94d/users/a4d1fbeb2b5345cc8a82bb6111c064bb/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:46.252729 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d6172-9525-4985-b1a9-bfa50573777b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e01be68d7d444f87bb454e140b04c94d, user_id=a4d1fbeb2b5345cc8a82bb6111c064bb, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:46.255442 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d6172-9525-4985-b1a9-bfa50573777b admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:46.275778 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1407/2798] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:42:46 2022] PUT /identity/v3/projects/e01be68d7d444f87bb454e140b04c94d/users/a4d1fbeb2b5345cc8a82bb6111c064bb/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:42:46.286107 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc18656-f72f-4f7b-8496-e311db01bfe2 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:46.286418 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc18656-f72f-4f7b-8496-e311db01bfe2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:46.286707 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc18656-f72f-4f7b-8496-e311db01bfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:46.321884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-9dc18656-f72f-4f7b-8496-e311db01bfe2 None None] MFA Rules not processed for user `a4d1fbeb2b5345cc8a82bb6111c064bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:46.324538 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9dc18656-f72f-4f7b-8496-e311db01bfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:46.366459 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:46.408559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1392/2799] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:46.413141 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d6172-9525-4985-b1a9-bfa50573777b admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:46.497427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d6172-9525-4985-b1a9-bfa50573777b admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:46.503951 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f23ef4-22fc-4ab5-be08-1f594f6ccb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:46.504640 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f23ef4-22fc-4ab5-be08-1f594f6ccb0f admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:46.504935 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f23ef4-22fc-4ab5-be08-1f594f6ccb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:46.505260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f23ef4-22fc-4ab5-be08-1f594f6ccb0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:46.506012 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f23ef4-22fc-4ab5-be08-1f594f6ccb0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:46.511827 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f23ef4-22fc-4ab5-be08-1f594f6ccb0f admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:46.550982 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1408/2800] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:42:46 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:42:46.554688 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc18656-f72f-4f7b-8496-e311db01bfe2 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:46.579598 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc18656-f72f-4f7b-8496-e311db01bfe2 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:46.692390 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4e8ad-b340-49f2-8eda-df6393517bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:46.693546 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4e8ad-b340-49f2-8eda-df6393517bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:46.693893 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4e8ad-b340-49f2-8eda-df6393517bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:46.694249 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4e8ad-b340-49f2-8eda-df6393517bd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:46.694968 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4e8ad-b340-49f2-8eda-df6393517bd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:46.697564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4e8ad-b340-49f2-8eda-df6393517bd8 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:46.725812 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1393/2801] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:42:46 2022] POST /identity/v3/users => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:42:46.730466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f23ef4-22fc-4ab5-be08-1f594f6ccb0f admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:46.745772 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f23ef4-22fc-4ab5-be08-1f594f6ccb0f admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:46.755604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e594d85-f1a8-4829-9b91-db88c3f74641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:46.756359 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e594d85-f1a8-4829-9b91-db88c3f74641 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:46.756685 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e594d85-f1a8-4829-9b91-db88c3f74641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:46.757114 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e594d85-f1a8-4829-9b91-db88c3f74641 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:46.757954 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e594d85-f1a8-4829-9b91-db88c3f74641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:46.759798 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e594d85-f1a8-4829-9b91-db88c3f74641 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:46.772396 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1409/2802] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:42:46.776621 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4e8ad-b340-49f2-8eda-df6393517bd8 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:46.788408 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4e8ad-b340-49f2-8eda-df6393517bd8 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:46.793470 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ac72a-db9a-4fd7-9659-67acca4b78b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:46.794389 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ac72a-db9a-4fd7-9659-67acca4b78b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:46.794684 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ac72a-db9a-4fd7-9659-67acca4b78b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:46.794968 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ac72a-db9a-4fd7-9659-67acca4b78b9 admin admin] PATH_INFO: `/v3/projects/5c0d0caf11d74e4282818981b937c5bd/users/1782f06e97d94125a86ec74210df03e6/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:46.820370 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ac72a-db9a-4fd7-9659-67acca4b78b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c0d0caf11d74e4282818981b937c5bd, user_id=1782f06e97d94125a86ec74210df03e6, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:46.823413 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ac72a-db9a-4fd7-9659-67acca4b78b9 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:46.845333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1394/2803] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:42:46 2022] PUT /identity/v3/projects/5c0d0caf11d74e4282818981b937c5bd/users/1782f06e97d94125a86ec74210df03e6/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:42:46.849510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e594d85-f1a8-4829-9b91-db88c3f74641 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:46.863401 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e594d85-f1a8-4829-9b91-db88c3f74641 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:46.947277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39ae97-7f7f-4cdb-84d9-b14f2abf647d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:46.948278 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39ae97-7f7f-4cdb-84d9-b14f2abf647d admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:46.948572 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39ae97-7f7f-4cdb-84d9-b14f2abf647d admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:46.948859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39ae97-7f7f-4cdb-84d9-b14f2abf647d admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:46.949648 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39ae97-7f7f-4cdb-84d9-b14f2abf647d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:46.951335 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39ae97-7f7f-4cdb-84d9-b14f2abf647d admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:46.963779 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1410/2804] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:42:46.969102 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ac72a-db9a-4fd7-9659-67acca4b78b9 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:46.987930 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ac72a-db9a-4fd7-9659-67acca4b78b9 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:46.998072 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b677a-4cbd-4fe5-ba63-702338f3805f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:46.998977 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b677a-4cbd-4fe5-ba63-702338f3805f admin admin] REQUEST_METHOD: `PUT` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:46.999414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b677a-4cbd-4fe5-ba63-702338f3805f admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:47.000233 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b677a-4cbd-4fe5-ba63-702338f3805f admin admin] PATH_INFO: `/v3/projects/5c0d0caf11d74e4282818981b937c5bd/users/1782f06e97d94125a86ec74210df03e6/roles/25a16d8ea4084e9eacdebb7a4dcd2337` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:47.002906 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b677a-4cbd-4fe5-ba63-702338f3805f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c0d0caf11d74e4282818981b937c5bd, user_id=1782f06e97d94125a86ec74210df03e6, role_id=25a16d8ea4084e9eacdebb7a4dcd2337)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:47.005926 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b677a-4cbd-4fe5-ba63-702338f3805f admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:47.019981 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1395/2805] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:42:46 2022] PUT /identity/v3/projects/5c0d0caf11d74e4282818981b937c5bd/users/1782f06e97d94125a86ec74210df03e6/roles/25a16d8ea4084e9eacdebb7a4dcd2337 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:47.031331 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22e96b-f35b-470b-b249-5797a104c985 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:47.031703 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22e96b-f35b-470b-b249-5797a104c985 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:47.031923 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22e96b-f35b-470b-b249-5797a104c985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:47.065222 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-fc22e96b-f35b-470b-b249-5797a104c985 None None] MFA Rules not processed for user `1782f06e97d94125a86ec74210df03e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:47.066622 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-fc22e96b-f35b-470b-b249-5797a104c985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:47.115876 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:47.163793 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1411/2806] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:47.175275 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b677a-4cbd-4fe5-ba63-702338f3805f admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:47.177065 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-c27b677a-4cbd-4fe5-ba63-702338f3805f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:47.246979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b677a-4cbd-4fe5-ba63-702338f3805f admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:47.251875 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c47b4-bfab-4cae-9991-45d12d423888 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:47.252884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c47b4-bfab-4cae-9991-45d12d423888 service cinder] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:47.253356 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c47b4-bfab-4cae-9991-45d12d423888 service cinder] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:47.253644 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c47b4-bfab-4cae-9991-45d12d423888 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:47.264021 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c47b4-bfab-4cae-9991-45d12d423888 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:47.266092 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c47b4-bfab-4cae-9991-45d12d423888 service cinder] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:47.276085 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1396/2807] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:42:47.352406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22e96b-f35b-470b-b249-5797a104c985 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:47.363294 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22e96b-f35b-470b-b249-5797a104c985 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:47.367139 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba330c-3b11-4c3b-8fb2-b429bcc3069b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:47.367815 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba330c-3b11-4c3b-8fb2-b429bcc3069b service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:47.368137 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba330c-3b11-4c3b-8fb2-b429bcc3069b service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:47.368420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba330c-3b11-4c3b-8fb2-b429bcc3069b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:47.429120 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba330c-3b11-4c3b-8fb2-b429bcc3069b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:47.430605 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba330c-3b11-4c3b-8fb2-b429bcc3069b service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:47.440590 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1412/2808] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:42:47.708586 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c47b4-bfab-4cae-9991-45d12d423888 service cinder] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:47.710106 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4f7c47b4-bfab-4cae-9991-45d12d423888 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:47.740199 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba330c-3b11-4c3b-8fb2-b429bcc3069b service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:47.752787 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:47.803339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c47b4-bfab-4cae-9991-45d12d423888 service cinder] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:47.810696 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922eea4-abab-4d9e-9249-03c09267cfe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:47.811862 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922eea4-abab-4d9e-9249-03c09267cfe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:47.812338 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922eea4-abab-4d9e-9249-03c09267cfe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:47.813059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922eea4-abab-4d9e-9249-03c09267cfe4 admin admin] PATH_INFO: `/v3/users/d5de893fcbb341168261082757384420` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:47.816222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922eea4-abab-4d9e-9249-03c09267cfe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5de893fcbb341168261082757384420)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:47.819954 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922eea4-abab-4d9e-9249-03c09267cfe4 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:47.837799 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba330c-3b11-4c3b-8fb2-b429bcc3069b service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:47.846462 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17151f44-5567-46b2-9449-12caa5e98b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:47.847865 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17151f44-5567-46b2-9449-12caa5e98b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:47.848353 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17151f44-5567-46b2-9449-12caa5e98b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:47.848848 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17151f44-5567-46b2-9449-12caa5e98b9b admin admin] PATH_INFO: `/v3/users/a4d1fbeb2b5345cc8a82bb6111c064bb` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:47.851596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17151f44-5567-46b2-9449-12caa5e98b9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4d1fbeb2b5345cc8a82bb6111c064bb)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:47.854244 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17151f44-5567-46b2-9449-12caa5e98b9b admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:47.899282 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5922eea4-abab-4d9e-9249-03c09267cfe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5de893fcbb341168261082757384420'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:47.915454 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5922eea4-abab-4d9e-9249-03c09267cfe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5de893fcbb341168261082757384420'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:47.916498 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5922eea4-abab-4d9e-9249-03c09267cfe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5de893fcbb341168261082757384420'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:47.926100 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5922eea4-abab-4d9e-9249-03c09267cfe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5de893fcbb341168261082757384420'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:47.929653 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-17151f44-5567-46b2-9449-12caa5e98b9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4d1fbeb2b5345cc8a82bb6111c064bb'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:47.936843 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-5922eea4-abab-4d9e-9249-03c09267cfe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5de893fcbb341168261082757384420'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:47.944346 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1397/2809] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:42:47 2022] DELETE /identity/v3/users/d5de893fcbb341168261082757384420 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:47.952407 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9b7f8-dcac-48d7-91bf-b174d6e35c2c None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:47.953497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9b7f8-dcac-48d7-91bf-b174d6e35c2c None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:47.953962 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9b7f8-dcac-48d7-91bf-b174d6e35c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:47.958771 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-17151f44-5567-46b2-9449-12caa5e98b9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4d1fbeb2b5345cc8a82bb6111c064bb'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:47.959917 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-17151f44-5567-46b2-9449-12caa5e98b9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4d1fbeb2b5345cc8a82bb6111c064bb'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:47.976108 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-17151f44-5567-46b2-9449-12caa5e98b9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4d1fbeb2b5345cc8a82bb6111c064bb'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:47.980627 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-d0b9b7f8-dcac-48d7-91bf-b174d6e35c2c None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:47.982417 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d0b9b7f8-dcac-48d7-91bf-b174d6e35c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:47.984734 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:48.006908 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-17151f44-5567-46b2-9449-12caa5e98b9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4d1fbeb2b5345cc8a82bb6111c064bb'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:48.017180 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1413/2810] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:42:47 2022] DELETE /identity/v3/users/a4d1fbeb2b5345cc8a82bb6111c064bb => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:48.023414 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e535a9-e02e-46e0-acbc-941e3eefb446 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:48.023715 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e535a9-e02e-46e0-acbc-941e3eefb446 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:48.023999 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e535a9-e02e-46e0-acbc-941e3eefb446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:48.046844 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-b7e535a9-e02e-46e0-acbc-941e3eefb446 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:48.048325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b7e535a9-e02e-46e0-acbc-941e3eefb446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:48.067451 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1398/2811] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:42:48.078393 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9b7f8-dcac-48d7-91bf-b174d6e35c2c None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:48.080181 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-d0b9b7f8-dcac-48d7-91bf-b174d6e35c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:48.142732 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1414/2812] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:42:48.158559 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e535a9-e02e-46e0-acbc-941e3eefb446 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:48.160289 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-b7e535a9-e02e-46e0-acbc-941e3eefb446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:48.196886 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:48.198909 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9b7f8-dcac-48d7-91bf-b174d6e35c2c None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:48.277402 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e535a9-e02e-46e0-acbc-941e3eefb446 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:48.284936 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb351c9-92e9-494f-b524-acc6a2e7869d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:48.286003 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb351c9-92e9-494f-b524-acc6a2e7869d service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:48.286494 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb351c9-92e9-494f-b524-acc6a2e7869d service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:48.287026 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb351c9-92e9-494f-b524-acc6a2e7869d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:48.305261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb351c9-92e9-494f-b524-acc6a2e7869d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:48.307131 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb351c9-92e9-494f-b524-acc6a2e7869d service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:48.326261 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1415/2813] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:42:48.335574 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397018bc-81a4-46b3-ad80-76e7eb949355 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:48.336264 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397018bc-81a4-46b3-ad80-76e7eb949355 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:48.336556 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397018bc-81a4-46b3-ad80-76e7eb949355 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:48.336836 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397018bc-81a4-46b3-ad80-76e7eb949355 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:48.349199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397018bc-81a4-46b3-ad80-76e7eb949355 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:48.351067 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397018bc-81a4-46b3-ad80-76e7eb949355 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:48.366774 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1399/2814] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:42:48.719387 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb351c9-92e9-494f-b524-acc6a2e7869d service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:48.723726 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-7fb351c9-92e9-494f-b524-acc6a2e7869d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:48.742518 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb351c9-92e9-494f-b524-acc6a2e7869d service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:48.750041 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfbbfb-4375-44be-9559-0848f298e024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:48.752453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfbbfb-4375-44be-9559-0848f298e024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:48.752988 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfbbfb-4375-44be-9559-0848f298e024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:48.753191 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfbbfb-4375-44be-9559-0848f298e024 admin admin] PATH_INFO: `/v3/projects/e01be68d7d444f87bb454e140b04c94d` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:48.754697 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfbbfb-4375-44be-9559-0848f298e024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e01be68d7d444f87bb454e140b04c94d)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:48.756820 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfbbfb-4375-44be-9559-0848f298e024 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:48.760442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397018bc-81a4-46b3-ad80-76e7eb949355 service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:48.762388 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-397018bc-81a4-46b3-ad80-76e7eb949355 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:48.779737 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397018bc-81a4-46b3-ad80-76e7eb949355 service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:48.788825 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9c206-1bbf-4b2a-a86d-6de5475c6e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:48.790114 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9c206-1bbf-4b2a-a86d-6de5475c6e86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:48.790367 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9c206-1bbf-4b2a-a86d-6de5475c6e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:48.790685 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9c206-1bbf-4b2a-a86d-6de5475c6e86 admin admin] PATH_INFO: `/v3/projects/039b7c844f5542f2a55b6d7e7be5aa96` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:48.793643 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9c206-1bbf-4b2a-a86d-6de5475c6e86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=039b7c844f5542f2a55b6d7e7be5aa96)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:48.795783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9c206-1bbf-4b2a-a86d-6de5475c6e86 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:48.825752 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c5dfbbfb-4375-44be-9559-0848f298e024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e01be68d7d444f87bb454e140b04c94d', 'request_id': 'req-c5dfbbfb-4375-44be-9559-0848f298e024'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:48.832755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c5dfbbfb-4375-44be-9559-0848f298e024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e01be68d7d444f87bb454e140b04c94d', 'request_id': 'req-c5dfbbfb-4375-44be-9559-0848f298e024'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:48.840278 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c5dfbbfb-4375-44be-9559-0848f298e024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e01be68d7d444f87bb454e140b04c94d', 'request_id': 'req-c5dfbbfb-4375-44be-9559-0848f298e024'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:48.856595 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c2f9c206-1bbf-4b2a-a86d-6de5475c6e86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '039b7c844f5542f2a55b6d7e7be5aa96', 'request_id': 'req-c2f9c206-1bbf-4b2a-a86d-6de5475c6e86'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:48.860975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c5dfbbfb-4375-44be-9559-0848f298e024 admin admin] The token cache is being invalidate because project e01be68d7d444f87bb454e140b04c94d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:42:48.861614 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c5dfbbfb-4375-44be-9559-0848f298e024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:48.862260 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c5dfbbfb-4375-44be-9559-0848f298e024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:48.863981 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1416/2815] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:42:48 2022] DELETE /identity/v3/projects/e01be68d7d444f87bb454e140b04c94d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:48.867808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c2f9c206-1bbf-4b2a-a86d-6de5475c6e86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '039b7c844f5542f2a55b6d7e7be5aa96', 'request_id': 'req-c2f9c206-1bbf-4b2a-a86d-6de5475c6e86'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:48.868239 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfbbfb-4375-44be-9559-0848f298e024 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:48.869850 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-c5dfbbfb-4375-44be-9559-0848f298e024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:48.875863 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c2f9c206-1bbf-4b2a-a86d-6de5475c6e86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '039b7c844f5542f2a55b6d7e7be5aa96', 'request_id': 'req-c2f9c206-1bbf-4b2a-a86d-6de5475c6e86'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:48.887702 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:48.889398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c2f9c206-1bbf-4b2a-a86d-6de5475c6e86 admin admin] The token cache is being invalidate because project 039b7c844f5542f2a55b6d7e7be5aa96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:42:48.889756 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c2f9c206-1bbf-4b2a-a86d-6de5475c6e86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:48.891317 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-c2f9c206-1bbf-4b2a-a86d-6de5475c6e86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:48.892923 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1400/2816] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:42:48 2022] DELETE /identity/v3/projects/039b7c844f5542f2a55b6d7e7be5aa96 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:42:48.897814 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a5efb-7dce-4f99-87e5-dcd8c7e3bd0f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:48.898183 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a5efb-7dce-4f99-87e5-dcd8c7e3bd0f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:48.898608 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a5efb-7dce-4f99-87e5-dcd8c7e3bd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:48.917698 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-fb4a5efb-7dce-4f99-87e5-dcd8c7e3bd0f None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:48.918937 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-fb4a5efb-7dce-4f99-87e5-dcd8c7e3bd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:48.950761 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfbbfb-4375-44be-9559-0848f298e024 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:48.985182 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1401/2817] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:48.995128 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f9889-07fd-4dbf-8c57-3767afd21aee None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:48.995595 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f9889-07fd-4dbf-8c57-3767afd21aee None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:48.996219 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f9889-07fd-4dbf-8c57-3767afd21aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:49.018514 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-4b8f9889-07fd-4dbf-8c57-3767afd21aee None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:49.019854 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-4b8f9889-07fd-4dbf-8c57-3767afd21aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:49.038851 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1402/2818] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:42:49.039145 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e425bb0-305d-4574-87f0-9b130d9d93a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:49.040448 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e425bb0-305d-4574-87f0-9b130d9d93a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:49.041249 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e425bb0-305d-4574-87f0-9b130d9d93a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:49.041886 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e425bb0-305d-4574-87f0-9b130d9d93a7 admin admin] PATH_INFO: `/v3/users/1782f06e97d94125a86ec74210df03e6` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:49.043031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f9889-07fd-4dbf-8c57-3767afd21aee None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:49.043330 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e425bb0-305d-4574-87f0-9b130d9d93a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1782f06e97d94125a86ec74210df03e6)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:49.046157 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e425bb0-305d-4574-87f0-9b130d9d93a7 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:49.054499 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f9889-07fd-4dbf-8c57-3767afd21aee None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:49.060185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b477d-b44e-441b-8618-9f3e191f0eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:49.060875 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b477d-b44e-441b-8618-9f3e191f0eef admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:49.061178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b477d-b44e-441b-8618-9f3e191f0eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:49.061482 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b477d-b44e-441b-8618-9f3e191f0eef admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:49.062808 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b477d-b44e-441b-8618-9f3e191f0eef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:42:49.063125 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b477d-b44e-441b-8618-9f3e191f0eef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:49.064745 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b477d-b44e-441b-8618-9f3e191f0eef admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:49.080502 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1403/2819] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:42:49.085660 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b477d-b44e-441b-8618-9f3e191f0eef admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:49.096604 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b477d-b44e-441b-8618-9f3e191f0eef admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:49.101719 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4363ab-aa17-4927-885b-a4f3423dffbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:49.102371 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4363ab-aa17-4927-885b-a4f3423dffbb admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:49.102665 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4363ab-aa17-4927-885b-a4f3423dffbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:49.102955 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4363ab-aa17-4927-885b-a4f3423dffbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:49.104063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4363ab-aa17-4927-885b-a4f3423dffbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:49.106063 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4363ab-aa17-4927-885b-a4f3423dffbb admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:49.111077 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:49.126537 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-7e425bb0-305d-4574-87f0-9b130d9d93a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1782f06e97d94125a86ec74210df03e6'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:49.143529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-7e425bb0-305d-4574-87f0-9b130d9d93a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1782f06e97d94125a86ec74210df03e6'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:49.144426 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-7e425bb0-305d-4574-87f0-9b130d9d93a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1782f06e97d94125a86ec74210df03e6'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:49.154088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-7e425bb0-305d-4574-87f0-9b130d9d93a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1782f06e97d94125a86ec74210df03e6'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:49.154411 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1404/2820] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:42:49 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:42:49.158683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4363ab-aa17-4927-885b-a4f3423dffbb admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:49.168075 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-7e425bb0-305d-4574-87f0-9b130d9d93a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1782f06e97d94125a86ec74210df03e6'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:49.175827 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1417/2821] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:42:48 2022] DELETE /identity/v3/users/1782f06e97d94125a86ec74210df03e6 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:42:49.245899 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4363ab-aa17-4927-885b-a4f3423dffbb admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:49.334327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e8f2f-2009-45e4-9c55-4754310b56bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:49.335446 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e8f2f-2009-45e4-9c55-4754310b56bc admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:49.335801 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e8f2f-2009-45e4-9c55-4754310b56bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:49.336208 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e8f2f-2009-45e4-9c55-4754310b56bc admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:49.336958 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e8f2f-2009-45e4-9c55-4754310b56bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:49.338956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e8f2f-2009-45e4-9c55-4754310b56bc admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:49.364339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1405/2822] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:42:49 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:42:49.368567 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e425bb0-305d-4574-87f0-9b130d9d93a7 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:49.370118 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-7e425bb0-305d-4574-87f0-9b130d9d93a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:49.385656 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e425bb0-305d-4574-87f0-9b130d9d93a7 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:49.393002 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70964cc-8f87-41ba-97e7-6773a911f3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:49.393826 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70964cc-8f87-41ba-97e7-6773a911f3e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:49.394186 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70964cc-8f87-41ba-97e7-6773a911f3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:49.394539 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70964cc-8f87-41ba-97e7-6773a911f3e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:49.395365 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70964cc-8f87-41ba-97e7-6773a911f3e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:49.397077 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70964cc-8f87-41ba-97e7-6773a911f3e2 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:49.408255 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1418/2823] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:42:49.412325 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e8f2f-2009-45e4-9c55-4754310b56bc admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:49.423377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e8f2f-2009-45e4-9c55-4754310b56bc admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:49.434335 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f63105-82ee-44be-a970-a29a96beec18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:49.435121 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f63105-82ee-44be-a970-a29a96beec18 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:49.438587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f63105-82ee-44be-a970-a29a96beec18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:49.438587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f63105-82ee-44be-a970-a29a96beec18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:49.438587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f63105-82ee-44be-a970-a29a96beec18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:49.439140 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f63105-82ee-44be-a970-a29a96beec18 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:49.452324 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1406/2824] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:42:49.458613 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70964cc-8f87-41ba-97e7-6773a911f3e2 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:49.486872 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70964cc-8f87-41ba-97e7-6773a911f3e2 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:49.492251 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923cdec5-1058-40ed-90d8-98605be4079f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:49.493278 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923cdec5-1058-40ed-90d8-98605be4079f admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:49.493579 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923cdec5-1058-40ed-90d8-98605be4079f admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:49.493845 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923cdec5-1058-40ed-90d8-98605be4079f admin admin] PATH_INFO: `/v3/projects/60629f0ce3364e018f570baf10a29a55/users/cb28e4ff119a431bba4a722c18f5a2a9/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:49.500815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f63105-82ee-44be-a970-a29a96beec18 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:49.503001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-36f63105-82ee-44be-a970-a29a96beec18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:49.519185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f63105-82ee-44be-a970-a29a96beec18 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:49.526016 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923cdec5-1058-40ed-90d8-98605be4079f admin admin] RBAC: Authorizing `identity:create_grant(project_id=60629f0ce3364e018f570baf10a29a55, user_id=cb28e4ff119a431bba4a722c18f5a2a9, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:49.529922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923cdec5-1058-40ed-90d8-98605be4079f admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:49.532513 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e361f5-2e62-4c1a-8fcc-02fad4cb6904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:49.533503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e361f5-2e62-4c1a-8fcc-02fad4cb6904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:49.533807 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e361f5-2e62-4c1a-8fcc-02fad4cb6904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:49.534122 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e361f5-2e62-4c1a-8fcc-02fad4cb6904 admin admin] PATH_INFO: `/v3/projects/5c0d0caf11d74e4282818981b937c5bd` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:49.536012 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e361f5-2e62-4c1a-8fcc-02fad4cb6904 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c0d0caf11d74e4282818981b937c5bd)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:49.538884 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e361f5-2e62-4c1a-8fcc-02fad4cb6904 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:49.559530 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1419/2825] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:42:49 2022] PUT /identity/v3/projects/60629f0ce3364e018f570baf10a29a55/users/cb28e4ff119a431bba4a722c18f5a2a9/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:42:49.571441 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ede28-321f-43ce-a8c4-f452e92a2929 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:49.571782 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ede28-321f-43ce-a8c4-f452e92a2929 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:49.572005 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ede28-321f-43ce-a8c4-f452e92a2929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:49.580847 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:49.612733 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-640ede28-321f-43ce-a8c4-f452e92a2929 None None] MFA Rules not processed for user `cb28e4ff119a431bba4a722c18f5a2a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:49.614046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-640ede28-321f-43ce-a8c4-f452e92a2929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:49.626717 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-34e361f5-2e62-4c1a-8fcc-02fad4cb6904 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c0d0caf11d74e4282818981b937c5bd', 'request_id': 'req-34e361f5-2e62-4c1a-8fcc-02fad4cb6904'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:49.634447 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-34e361f5-2e62-4c1a-8fcc-02fad4cb6904 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c0d0caf11d74e4282818981b937c5bd', 'request_id': 'req-34e361f5-2e62-4c1a-8fcc-02fad4cb6904'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:49.641398 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-34e361f5-2e62-4c1a-8fcc-02fad4cb6904 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c0d0caf11d74e4282818981b937c5bd', 'request_id': 'req-34e361f5-2e62-4c1a-8fcc-02fad4cb6904'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:49.647281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:49.657121 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-34e361f5-2e62-4c1a-8fcc-02fad4cb6904 admin admin] The token cache is being invalidate because project 5c0d0caf11d74e4282818981b937c5bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:42:49.657445 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-34e361f5-2e62-4c1a-8fcc-02fad4cb6904 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:49.658790 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-34e361f5-2e62-4c1a-8fcc-02fad4cb6904 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:42:49.661690 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1407/2826] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:42:49 2022] DELETE /identity/v3/projects/5c0d0caf11d74e4282818981b937c5bd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:49.669939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b266e92-2ab3-4f3d-94cc-5a3ae7bfa5e4 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:49.670377 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b266e92-2ab3-4f3d-94cc-5a3ae7bfa5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:49.670861 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b266e92-2ab3-4f3d-94cc-5a3ae7bfa5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:49.691590 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1420/2827] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:49.699336 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0b266e92-2ab3-4f3d-94cc-5a3ae7bfa5e4 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:49.700503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0b266e92-2ab3-4f3d-94cc-5a3ae7bfa5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:49.707476 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ede28-321f-43ce-a8c4-f452e92a2929 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:49.710389 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-640ede28-321f-43ce-a8c4-f452e92a2929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:49.774903 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1408/2828] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:49.782888 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd0a7a-a411-4f7c-b706-26707a0d4dd0 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:49.783211 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd0a7a-a411-4f7c-b706-26707a0d4dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:49.783516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd0a7a-a411-4f7c-b706-26707a0d4dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:49.795163 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ede28-321f-43ce-a8c4-f452e92a2929 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:49.803857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29e8bc-4fed-4c17-98b1-6940ba4c2565 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:49.803857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29e8bc-4fed-4c17-98b1-6940ba4c2565 service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:49.803857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29e8bc-4fed-4c17-98b1-6940ba4c2565 service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:49.803857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29e8bc-4fed-4c17-98b1-6940ba4c2565 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:49.809742 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-e9fd0a7a-a411-4f7c-b706-26707a0d4dd0 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:49.811001 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e9fd0a7a-a411-4f7c-b706-26707a0d4dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:49.818652 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29e8bc-4fed-4c17-98b1-6940ba4c2565 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:49.820126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29e8bc-4fed-4c17-98b1-6940ba4c2565 service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:49.820532 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1409/2829] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:42:49.824978 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd0a7a-a411-4f7c-b706-26707a0d4dd0 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:49.832491 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1421/2830] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:42:49.834847 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd0a7a-a411-4f7c-b706-26707a0d4dd0 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:49.839873 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e31308-27df-4a20-9a49-f68494f82608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:49.840503 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e31308-27df-4a20-9a49-f68494f82608 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:49.840794 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e31308-27df-4a20-9a49-f68494f82608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:49.841079 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e31308-27df-4a20-9a49-f68494f82608 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:49.841858 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e31308-27df-4a20-9a49-f68494f82608 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:42:49.842178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e31308-27df-4a20-9a49-f68494f82608 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:49.844339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e31308-27df-4a20-9a49-f68494f82608 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:49.857632 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1410/2831] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:42:49.863404 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29e8bc-4fed-4c17-98b1-6940ba4c2565 service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:49.875327 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29e8bc-4fed-4c17-98b1-6940ba4c2565 service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:49.880256 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8642b27-8f6d-4eb0-8283-c22c2280b774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:49.880933 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8642b27-8f6d-4eb0-8283-c22c2280b774 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:49.881275 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8642b27-8f6d-4eb0-8283-c22c2280b774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:49.881645 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8642b27-8f6d-4eb0-8283-c22c2280b774 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:49.882389 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8642b27-8f6d-4eb0-8283-c22c2280b774 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:49.884403 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8642b27-8f6d-4eb0-8283-c22c2280b774 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:49.923239 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1422/2832] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:42:49 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:42:49.924893 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e31308-27df-4a20-9a49-f68494f82608 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:49.936658 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e31308-27df-4a20-9a49-f68494f82608 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:50.008133 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655fa33-a9ee-47e1-b6c4-ca1df0d4363d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:50.008822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655fa33-a9ee-47e1-b6c4-ca1df0d4363d admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:50.009108 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655fa33-a9ee-47e1-b6c4-ca1df0d4363d admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:50.009422 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655fa33-a9ee-47e1-b6c4-ca1df0d4363d admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:50.010216 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655fa33-a9ee-47e1-b6c4-ca1df0d4363d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:50.012159 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655fa33-a9ee-47e1-b6c4-ca1df0d4363d admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:50.042987 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1411/2833] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:42:49 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:42:50.044673 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8642b27-8f6d-4eb0-8283-c22c2280b774 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:50.058821 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8642b27-8f6d-4eb0-8283-c22c2280b774 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:50.065234 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb2c77-5d29-4926-b808-095c8619696a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:50.065941 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb2c77-5d29-4926-b808-095c8619696a admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:50.067259 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb2c77-5d29-4926-b808-095c8619696a admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:50.067582 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb2c77-5d29-4926-b808-095c8619696a admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:50.068446 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb2c77-5d29-4926-b808-095c8619696a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:50.070612 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb2c77-5d29-4926-b808-095c8619696a admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:50.083748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1423/2834] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:42:50.094049 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655fa33-a9ee-47e1-b6c4-ca1df0d4363d admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:50.108164 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655fa33-a9ee-47e1-b6c4-ca1df0d4363d admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:50.115175 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e0e1c-56e4-436e-80cf-05a1b5c4e7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:50.116290 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e0e1c-56e4-436e-80cf-05a1b5c4e7aa admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:50.118578 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e0e1c-56e4-436e-80cf-05a1b5c4e7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:50.119236 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e0e1c-56e4-436e-80cf-05a1b5c4e7aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:50.120791 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e0e1c-56e4-436e-80cf-05a1b5c4e7aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:50.123914 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e0e1c-56e4-436e-80cf-05a1b5c4e7aa admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:50.137007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1412/2835] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:42:50.142737 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb2c77-5d29-4926-b808-095c8619696a admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:50.158008 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb2c77-5d29-4926-b808-095c8619696a admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:50.170752 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf9dfe-3f7c-44a6-bd06-8d6b5b52f4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:50.172391 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf9dfe-3f7c-44a6-bd06-8d6b5b52f4b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:50.172748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf9dfe-3f7c-44a6-bd06-8d6b5b52f4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:50.173033 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf9dfe-3f7c-44a6-bd06-8d6b5b52f4b0 admin admin] PATH_INFO: `/v3/projects/e1e5cf8fe9c04e32aac0dd642c4cd4fe/users/8e667007d85b466387e16dde9c8980bc/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:50.191701 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf9dfe-3f7c-44a6-bd06-8d6b5b52f4b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1e5cf8fe9c04e32aac0dd642c4cd4fe, user_id=8e667007d85b466387e16dde9c8980bc, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:50.194776 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf9dfe-3f7c-44a6-bd06-8d6b5b52f4b0 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:50.218596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1424/2836] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:42:50 2022] PUT /identity/v3/projects/e1e5cf8fe9c04e32aac0dd642c4cd4fe/users/8e667007d85b466387e16dde9c8980bc/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:42:50.234560 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329ecab-163b-4959-b681-233a3b19a8a7 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:50.234560 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329ecab-163b-4959-b681-233a3b19a8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:50.234560 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329ecab-163b-4959-b681-233a3b19a8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:50.285975 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-0329ecab-163b-4959-b681-233a3b19a8a7 None None] MFA Rules not processed for user `8e667007d85b466387e16dde9c8980bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:50.288008 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-0329ecab-163b-4959-b681-233a3b19a8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:50.361956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:50.386419 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1413/2837] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:42:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:50.398337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf9dfe-3f7c-44a6-bd06-8d6b5b52f4b0 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:50.445174 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:50.461716 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf9dfe-3f7c-44a6-bd06-8d6b5b52f4b0 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:50.465368 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39286dc-0712-466a-9955-e8b2c42bfba1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:50.466017 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39286dc-0712-466a-9955-e8b2c42bfba1 service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:50.466306 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39286dc-0712-466a-9955-e8b2c42bfba1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:50.466607 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39286dc-0712-466a-9955-e8b2c42bfba1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:50.475920 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39286dc-0712-466a-9955-e8b2c42bfba1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:50.477566 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39286dc-0712-466a-9955-e8b2c42bfba1 service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:50.487551 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1425/2838] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:43:00.383516 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=7 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:00.482934 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=9 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:03.274519 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329ecab-163b-4959-b681-233a3b19a8a7 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:03.343801 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329ecab-163b-4959-b681-233a3b19a8a7 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:03.348566 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288de937-7840-4ee3-9a29-6c7ca795931a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:03.349569 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288de937-7840-4ee3-9a29-6c7ca795931a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:03.349924 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288de937-7840-4ee3-9a29-6c7ca795931a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:03.350268 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288de937-7840-4ee3-9a29-6c7ca795931a admin admin] PATH_INFO: `/v3/users/8e667007d85b466387e16dde9c8980bc` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:03.351553 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288de937-7840-4ee3-9a29-6c7ca795931a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e667007d85b466387e16dde9c8980bc)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:03.353925 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288de937-7840-4ee3-9a29-6c7ca795931a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:03.407205 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-288de937-7840-4ee3-9a29-6c7ca795931a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e667007d85b466387e16dde9c8980bc'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:03.419725 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-288de937-7840-4ee3-9a29-6c7ca795931a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e667007d85b466387e16dde9c8980bc'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:03.420494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-288de937-7840-4ee3-9a29-6c7ca795931a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e667007d85b466387e16dde9c8980bc'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:03.428259 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-288de937-7840-4ee3-9a29-6c7ca795931a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e667007d85b466387e16dde9c8980bc'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:03.438706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-288de937-7840-4ee3-9a29-6c7ca795931a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e667007d85b466387e16dde9c8980bc'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:03.445072 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1414/2839] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:43:03 2022] DELETE /identity/v3/users/8e667007d85b466387e16dde9c8980bc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:03.449610 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590de18-e9ac-49a1-ad84-f6f83d681d84 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:03.450089 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590de18-e9ac-49a1-ad84-f6f83d681d84 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:03.450190 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590de18-e9ac-49a1-ad84-f6f83d681d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:03.469620 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-f590de18-e9ac-49a1-ad84-f6f83d681d84 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:03.470857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-f590de18-e9ac-49a1-ad84-f6f83d681d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:03.537369 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1426/2840] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:03.547399 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288de937-7840-4ee3-9a29-6c7ca795931a admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:03.548825 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-288de937-7840-4ee3-9a29-6c7ca795931a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:03.645053 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288de937-7840-4ee3-9a29-6c7ca795931a admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:03.651223 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d42844b-5fe3-4bcd-b8eb-4186e0a5c2a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:03.652249 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42844b-5fe3-4bcd-b8eb-4186e0a5c2a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:03.652727 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42844b-5fe3-4bcd-b8eb-4186e0a5c2a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:03.653213 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42844b-5fe3-4bcd-b8eb-4186e0a5c2a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:03.668640 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d42844b-5fe3-4bcd-b8eb-4186e0a5c2a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:03.670684 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d42844b-5fe3-4bcd-b8eb-4186e0a5c2a2 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:03.682778 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1415/2841] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:43:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:43:04.034215 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590de18-e9ac-49a1-ad84-f6f83d681d84 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:04.035812 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-f590de18-e9ac-49a1-ad84-f6f83d681d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:04.056779 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590de18-e9ac-49a1-ad84-f6f83d681d84 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:04.065332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f84466-e349-4f8c-9899-2591ae7ab8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:04.067000 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f84466-e349-4f8c-9899-2591ae7ab8e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:04.067537 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f84466-e349-4f8c-9899-2591ae7ab8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:04.068092 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f84466-e349-4f8c-9899-2591ae7ab8e8 admin admin] PATH_INFO: `/v3/projects/e1e5cf8fe9c04e32aac0dd642c4cd4fe` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:04.070416 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f84466-e349-4f8c-9899-2591ae7ab8e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1e5cf8fe9c04e32aac0dd642c4cd4fe)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:04.074414 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f84466-e349-4f8c-9899-2591ae7ab8e8 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:04.138046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a5f84466-e349-4f8c-9899-2591ae7ab8e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1e5cf8fe9c04e32aac0dd642c4cd4fe', 'request_id': 'req-a5f84466-e349-4f8c-9899-2591ae7ab8e8'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:04.144859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a5f84466-e349-4f8c-9899-2591ae7ab8e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1e5cf8fe9c04e32aac0dd642c4cd4fe', 'request_id': 'req-a5f84466-e349-4f8c-9899-2591ae7ab8e8'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:04.152222 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a5f84466-e349-4f8c-9899-2591ae7ab8e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1e5cf8fe9c04e32aac0dd642c4cd4fe', 'request_id': 'req-a5f84466-e349-4f8c-9899-2591ae7ab8e8'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:04.166128 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a5f84466-e349-4f8c-9899-2591ae7ab8e8 admin admin] The token cache is being invalidate because project e1e5cf8fe9c04e32aac0dd642c4cd4fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:43:04.166494 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a5f84466-e349-4f8c-9899-2591ae7ab8e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:04.167176 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a5f84466-e349-4f8c-9899-2591ae7ab8e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:04.168757 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1427/2842] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:43:04 2022] DELETE /identity/v3/projects/e1e5cf8fe9c04e32aac0dd642c4cd4fe => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:04.174087 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24100945-eb9a-4bf3-9a9f-453e5776496f None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:04.174388 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24100945-eb9a-4bf3-9a9f-453e5776496f None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:04.174675 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24100945-eb9a-4bf3-9a9f-453e5776496f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:04.194595 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-24100945-eb9a-4bf3-9a9f-453e5776496f None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:04.195914 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-24100945-eb9a-4bf3-9a9f-453e5776496f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:04.219872 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:04.282759 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1416/2843] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:43:04.291673 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9cf8d-acd9-48e0-bc1d-b432df56b56e None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:04.292187 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9cf8d-acd9-48e0-bc1d-b432df56b56e None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:04.292576 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9cf8d-acd9-48e0-bc1d-b432df56b56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:04.312622 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-47a9cf8d-acd9-48e0-bc1d-b432df56b56e None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:04.316406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-47a9cf8d-acd9-48e0-bc1d-b432df56b56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:04.322493 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1428/2844] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:04.327439 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24100945-eb9a-4bf3-9a9f-453e5776496f None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:04.337623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24100945-eb9a-4bf3-9a9f-453e5776496f None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:04.343673 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964aeeff-ec56-4913-a6b9-a272daac8d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:04.344359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964aeeff-ec56-4913-a6b9-a272daac8d18 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:04.344653 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964aeeff-ec56-4913-a6b9-a272daac8d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:04.344939 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964aeeff-ec56-4913-a6b9-a272daac8d18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:04.345730 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964aeeff-ec56-4913-a6b9-a272daac8d18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:43:04.346051 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964aeeff-ec56-4913-a6b9-a272daac8d18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:04.347656 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964aeeff-ec56-4913-a6b9-a272daac8d18 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:04.361510 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1417/2845] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:43:04.365888 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9cf8d-acd9-48e0-bc1d-b432df56b56e None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:04.375435 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9cf8d-acd9-48e0-bc1d-b432df56b56e None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:04.380288 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26408805-edf9-4ca8-8b3d-13fbfbfc1dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:04.380959 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26408805-edf9-4ca8-8b3d-13fbfbfc1dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:04.381297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26408805-edf9-4ca8-8b3d-13fbfbfc1dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:04.381585 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26408805-edf9-4ca8-8b3d-13fbfbfc1dc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:04.382417 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26408805-edf9-4ca8-8b3d-13fbfbfc1dc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:04.384835 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26408805-edf9-4ca8-8b3d-13fbfbfc1dc6 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:04.415432 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1429/2846] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:43:04 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:04.419721 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964aeeff-ec56-4913-a6b9-a272daac8d18 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:04.430651 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964aeeff-ec56-4913-a6b9-a272daac8d18 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:04.493027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a8754-39b2-4bd0-bbf1-9339e10a14f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:04.493764 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a8754-39b2-4bd0-bbf1-9339e10a14f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:04.494059 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a8754-39b2-4bd0-bbf1-9339e10a14f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:04.494340 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a8754-39b2-4bd0-bbf1-9339e10a14f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:04.495043 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a8754-39b2-4bd0-bbf1-9339e10a14f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:04.496979 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a8754-39b2-4bd0-bbf1-9339e10a14f2 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:04.520010 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1418/2847] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:43:04 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:04.524747 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26408805-edf9-4ca8-8b3d-13fbfbfc1dc6 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:04.536252 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26408805-edf9-4ca8-8b3d-13fbfbfc1dc6 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:04.541109 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146cce6-0c62-4f80-a412-9573abdeabf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:04.541814 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146cce6-0c62-4f80-a412-9573abdeabf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:04.542104 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146cce6-0c62-4f80-a412-9573abdeabf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:04.542387 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146cce6-0c62-4f80-a412-9573abdeabf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:04.543146 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e146cce6-0c62-4f80-a412-9573abdeabf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:04.544755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e146cce6-0c62-4f80-a412-9573abdeabf0 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:04.557025 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1430/2848] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:04.561102 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a8754-39b2-4bd0-bbf1-9339e10a14f2 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:04.573133 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a8754-39b2-4bd0-bbf1-9339e10a14f2 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:04.578521 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a95b5-2e08-4bc7-8e2f-4f073658bb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:04.579222 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a95b5-2e08-4bc7-8e2f-4f073658bb4a admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:04.579522 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a95b5-2e08-4bc7-8e2f-4f073658bb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:04.579815 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a95b5-2e08-4bc7-8e2f-4f073658bb4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:04.580643 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a95b5-2e08-4bc7-8e2f-4f073658bb4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:04.582344 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a95b5-2e08-4bc7-8e2f-4f073658bb4a admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:04.594527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1419/2849] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:04.598675 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146cce6-0c62-4f80-a412-9573abdeabf0 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:04.609866 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146cce6-0c62-4f80-a412-9573abdeabf0 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:04.614813 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a449de0-ef59-4aee-b3c3-e82885681067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:04.615730 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a449de0-ef59-4aee-b3c3-e82885681067 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:04.616025 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a449de0-ef59-4aee-b3c3-e82885681067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:04.616313 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a449de0-ef59-4aee-b3c3-e82885681067 admin admin] PATH_INFO: `/v3/projects/54ad4014673546f8865a7659ad0b6ead/users/def8c33cd3ca4acea3fa85d533dfcd91/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:04.633349 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a449de0-ef59-4aee-b3c3-e82885681067 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54ad4014673546f8865a7659ad0b6ead, user_id=def8c33cd3ca4acea3fa85d533dfcd91, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:04.636019 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a449de0-ef59-4aee-b3c3-e82885681067 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:04.650047 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1431/2850] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:43:04 2022] PUT /identity/v3/projects/54ad4014673546f8865a7659ad0b6ead/users/def8c33cd3ca4acea3fa85d533dfcd91/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:04.660662 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afdb6f9-a8bd-459d-89c1-0b71451a1191 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:04.660960 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afdb6f9-a8bd-459d-89c1-0b71451a1191 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:04.661307 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afdb6f9-a8bd-459d-89c1-0b71451a1191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:04.690736 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-9afdb6f9-a8bd-459d-89c1-0b71451a1191 None None] MFA Rules not processed for user `def8c33cd3ca4acea3fa85d533dfcd91`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:04.692095 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9afdb6f9-a8bd-459d-89c1-0b71451a1191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:04.762621 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1420/2851] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:43:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:04.774968 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a449de0-ef59-4aee-b3c3-e82885681067 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:04.776723 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-8a449de0-ef59-4aee-b3c3-e82885681067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:04.800887 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:04.838577 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a449de0-ef59-4aee-b3c3-e82885681067 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:04.842921 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11882931-6b49-4c36-b709-0343f8163ebd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:04.843611 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11882931-6b49-4c36-b709-0343f8163ebd service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:04.843905 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11882931-6b49-4c36-b709-0343f8163ebd service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:04.844354 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11882931-6b49-4c36-b709-0343f8163ebd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:04.854126 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11882931-6b49-4c36-b709-0343f8163ebd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:04.855489 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11882931-6b49-4c36-b709-0343f8163ebd service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:04.865522 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1432/2852] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:43:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:43:04.891359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afdb6f9-a8bd-459d-89c1-0b71451a1191 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:04.912668 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:04.960608 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afdb6f9-a8bd-459d-89c1-0b71451a1191 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:04.965466 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131777b7-f749-4697-b87c-20d8b283df2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:04.966353 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131777b7-f749-4697-b87c-20d8b283df2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:04.966639 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131777b7-f749-4697-b87c-20d8b283df2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:04.966915 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131777b7-f749-4697-b87c-20d8b283df2e admin admin] PATH_INFO: `/v3/users/def8c33cd3ca4acea3fa85d533dfcd91` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:04.968334 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131777b7-f749-4697-b87c-20d8b283df2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=def8c33cd3ca4acea3fa85d533dfcd91)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:04.970309 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131777b7-f749-4697-b87c-20d8b283df2e admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:05.019961 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-131777b7-f749-4697-b87c-20d8b283df2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'def8c33cd3ca4acea3fa85d533dfcd91'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:05.034003 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-131777b7-f749-4697-b87c-20d8b283df2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'def8c33cd3ca4acea3fa85d533dfcd91'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:05.034742 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-131777b7-f749-4697-b87c-20d8b283df2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'def8c33cd3ca4acea3fa85d533dfcd91'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:05.042367 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-131777b7-f749-4697-b87c-20d8b283df2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'def8c33cd3ca4acea3fa85d533dfcd91'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:05.051321 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-131777b7-f749-4697-b87c-20d8b283df2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'def8c33cd3ca4acea3fa85d533dfcd91'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:05.058075 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1421/2853] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:43:04 2022] DELETE /identity/v3/users/def8c33cd3ca4acea3fa85d533dfcd91 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:05.063315 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb7dcf-1a21-41bf-8c1b-0a496cd05b5e None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:05.063616 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb7dcf-1a21-41bf-8c1b-0a496cd05b5e None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:05.063926 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb7dcf-1a21-41bf-8c1b-0a496cd05b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:05.081903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-77fb7dcf-1a21-41bf-8c1b-0a496cd05b5e None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:05.083107 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-77fb7dcf-1a21-41bf-8c1b-0a496cd05b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:05.159363 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1433/2854] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:05.169119 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131777b7-f749-4697-b87c-20d8b283df2e admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:05.171548 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-131777b7-f749-4697-b87c-20d8b283df2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:05.247046 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131777b7-f749-4697-b87c-20d8b283df2e admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:05.251379 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d61a38-4864-4e9b-8f4f-ef099c6e7919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:05.252040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d61a38-4864-4e9b-8f4f-ef099c6e7919 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:05.252326 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d61a38-4864-4e9b-8f4f-ef099c6e7919 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:05.252635 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d61a38-4864-4e9b-8f4f-ef099c6e7919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:05.262264 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d61a38-4864-4e9b-8f4f-ef099c6e7919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:05.263695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d61a38-4864-4e9b-8f4f-ef099c6e7919 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:05.274040 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1422/2855] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:43:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:43:05.581902 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb7dcf-1a21-41bf-8c1b-0a496cd05b5e None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:05.583468 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-77fb7dcf-1a21-41bf-8c1b-0a496cd05b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:05.598643 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb7dcf-1a21-41bf-8c1b-0a496cd05b5e None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:05.604462 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb8eb7-d1ef-42b9-a8c8-b0550cdf8e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:05.605562 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb8eb7-d1ef-42b9-a8c8-b0550cdf8e5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:05.605859 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb8eb7-d1ef-42b9-a8c8-b0550cdf8e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:05.606144 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb8eb7-d1ef-42b9-a8c8-b0550cdf8e5b admin admin] PATH_INFO: `/v3/projects/54ad4014673546f8865a7659ad0b6ead` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:05.607400 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb8eb7-d1ef-42b9-a8c8-b0550cdf8e5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=54ad4014673546f8865a7659ad0b6ead)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:05.610138 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb8eb7-d1ef-42b9-a8c8-b0550cdf8e5b admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:05.667607 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9fcb8eb7-d1ef-42b9-a8c8-b0550cdf8e5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54ad4014673546f8865a7659ad0b6ead', 'request_id': 'req-9fcb8eb7-d1ef-42b9-a8c8-b0550cdf8e5b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:05.675591 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9fcb8eb7-d1ef-42b9-a8c8-b0550cdf8e5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54ad4014673546f8865a7659ad0b6ead', 'request_id': 'req-9fcb8eb7-d1ef-42b9-a8c8-b0550cdf8e5b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:05.683102 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9fcb8eb7-d1ef-42b9-a8c8-b0550cdf8e5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54ad4014673546f8865a7659ad0b6ead', 'request_id': 'req-9fcb8eb7-d1ef-42b9-a8c8-b0550cdf8e5b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:05.696768 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9fcb8eb7-d1ef-42b9-a8c8-b0550cdf8e5b admin admin] The token cache is being invalidate because project 54ad4014673546f8865a7659ad0b6ead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:43:05.697071 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9fcb8eb7-d1ef-42b9-a8c8-b0550cdf8e5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:05.697624 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-9fcb8eb7-d1ef-42b9-a8c8-b0550cdf8e5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:05.699215 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1434/2856] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:43:05 2022] DELETE /identity/v3/projects/54ad4014673546f8865a7659ad0b6ead => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:05.705238 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c8d41-54e7-40bf-ba66-acd5c8be8e23 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:05.706029 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c8d41-54e7-40bf-ba66-acd5c8be8e23 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:05.706215 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c8d41-54e7-40bf-ba66-acd5c8be8e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:05.726830 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-9a3c8d41-54e7-40bf-ba66-acd5c8be8e23 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:05.728338 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-9a3c8d41-54e7-40bf-ba66-acd5c8be8e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:05.768942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:05.803333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1423/2857] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:05.813080 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e5bdf-2aef-45be-8f35-c1d2b6ac530e None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:05.813594 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e5bdf-2aef-45be-8f35-c1d2b6ac530e None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:05.813970 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e5bdf-2aef-45be-8f35-c1d2b6ac530e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:05.834750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-136e5bdf-2aef-45be-8f35-c1d2b6ac530e None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:05.836088 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-136e5bdf-2aef-45be-8f35-c1d2b6ac530e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:05.849376 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1435/2858] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:05.856012 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c8d41-54e7-40bf-ba66-acd5c8be8e23 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:05.866584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c8d41-54e7-40bf-ba66-acd5c8be8e23 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:05.872352 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1bf871-ad94-45f7-bd8e-5e5ad159dd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:05.873695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bf871-ad94-45f7-bd8e-5e5ad159dd05 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:05.873695 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bf871-ad94-45f7-bd8e-5e5ad159dd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:05.874386 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bf871-ad94-45f7-bd8e-5e5ad159dd05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:05.875033 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1bf871-ad94-45f7-bd8e-5e5ad159dd05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80151) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:43:05.875494 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1bf871-ad94-45f7-bd8e-5e5ad159dd05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:05.877197 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1bf871-ad94-45f7-bd8e-5e5ad159dd05 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:05.894661 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1424/2859] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 17:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:43:05.899214 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e5bdf-2aef-45be-8f35-c1d2b6ac530e None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:05.911472 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e5bdf-2aef-45be-8f35-c1d2b6ac530e None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:05.918247 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe5ea5-dcc2-4277-9ab9-3eb5054088ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:05.919055 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe5ea5-dcc2-4277-9ab9-3eb5054088ea admin admin] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:05.919369 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe5ea5-dcc2-4277-9ab9-3eb5054088ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:05.919654 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe5ea5-dcc2-4277-9ab9-3eb5054088ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:05.920570 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe5ea5-dcc2-4277-9ab9-3eb5054088ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:05.923176 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe5ea5-dcc2-4277-9ab9-3eb5054088ea admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:05.954584 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1436/2860] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 17:43:05 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:43:05.959157 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1bf871-ad94-45f7-bd8e-5e5ad159dd05 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:05.976527 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1bf871-ad94-45f7-bd8e-5e5ad159dd05 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:06.042546 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12faf9-cbfa-4b82-800a-ab86cc9824fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:06.043236 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12faf9-cbfa-4b82-800a-ab86cc9824fa admin admin] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:06.043522 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12faf9-cbfa-4b82-800a-ab86cc9824fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:06.043802 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12faf9-cbfa-4b82-800a-ab86cc9824fa admin admin] PATH_INFO: `/v3/users` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:06.044497 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12faf9-cbfa-4b82-800a-ab86cc9824fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:06.046611 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12faf9-cbfa-4b82-800a-ab86cc9824fa admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:06.071552 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1425/2861] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 17:43:05 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:06.075655 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe5ea5-dcc2-4277-9ab9-3eb5054088ea admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:06.088070 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe5ea5-dcc2-4277-9ab9-3eb5054088ea admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:06.093214 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3e19c-aad3-4235-9305-10043e7f1dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:06.093909 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3e19c-aad3-4235-9305-10043e7f1dac admin admin] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:06.094205 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3e19c-aad3-4235-9305-10043e7f1dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:06.094492 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3e19c-aad3-4235-9305-10043e7f1dac admin admin] PATH_INFO: `/v3/roles` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:06.095240 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3e19c-aad3-4235-9305-10043e7f1dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:06.096861 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3e19c-aad3-4235-9305-10043e7f1dac admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:06.108474 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1437/2862] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:06.112660 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12faf9-cbfa-4b82-800a-ab86cc9824fa admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:06.123860 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12faf9-cbfa-4b82-800a-ab86cc9824fa admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:06.128931 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a9b01-ed0f-4393-a93a-43eb5b16ee53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:06.129655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a9b01-ed0f-4393-a93a-43eb5b16ee53 admin admin] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:06.129951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a9b01-ed0f-4393-a93a-43eb5b16ee53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:06.130237 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a9b01-ed0f-4393-a93a-43eb5b16ee53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:06.131054 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3a9b01-ed0f-4393-a93a-43eb5b16ee53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:06.132582 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3a9b01-ed0f-4393-a93a-43eb5b16ee53 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:06.144118 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1426/2863] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:06.148683 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3e19c-aad3-4235-9305-10043e7f1dac admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:06.159751 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3e19c-aad3-4235-9305-10043e7f1dac admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:06.164374 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe63cb6-2f85-495c-b612-4b0001630e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:06.165458 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe63cb6-2f85-495c-b612-4b0001630e82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:06.165458 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe63cb6-2f85-495c-b612-4b0001630e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:06.166436 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe63cb6-2f85-495c-b612-4b0001630e82 admin admin] PATH_INFO: `/v3/projects/2d208bc7521b47589bc3bcf5394fd361/users/3fb39b017d944d7d9808649e85903d43/roles/483146fdc4ac47cf97c04a9f66c091b4` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:06.168328 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:06.185332 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe63cb6-2f85-495c-b612-4b0001630e82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d208bc7521b47589bc3bcf5394fd361, user_id=3fb39b017d944d7d9808649e85903d43, role_id=483146fdc4ac47cf97c04a9f66c091b4)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:06.189410 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe63cb6-2f85-495c-b612-4b0001630e82 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:06.207910 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1438/2864] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 17:43:06 2022] PUT /identity/v3/projects/2d208bc7521b47589bc3bcf5394fd361/users/3fb39b017d944d7d9808649e85903d43/roles/483146fdc4ac47cf97c04a9f66c091b4 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:06.218596 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9692b-5c5e-45a4-88bd-29b82fdeb957 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:06.218904 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9692b-5c5e-45a4-88bd-29b82fdeb957 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:06.219156 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9692b-5c5e-45a4-88bd-29b82fdeb957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:06.263654 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-2dc9692b-5c5e-45a4-88bd-29b82fdeb957 None None] MFA Rules not processed for user `3fb39b017d944d7d9808649e85903d43`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:06.266013 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-2dc9692b-5c5e-45a4-88bd-29b82fdeb957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:06.398295 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1427/2865] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:43:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:06.412012 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe63cb6-2f85-495c-b612-4b0001630e82 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:06.413890 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-8fe63cb6-2f85-495c-b612-4b0001630e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:06.484307 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe63cb6-2f85-495c-b612-4b0001630e82 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:06.487943 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded38a73-245d-404a-9f46-31490b826866 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fadecafbe2549acaa4cde5412638d8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:06.488602 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded38a73-245d-404a-9f46-31490b826866 service cinder] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:06.489033 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded38a73-245d-404a-9f46-31490b826866 service cinder] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:06.489351 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded38a73-245d-404a-9f46-31490b826866 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:06.499393 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded38a73-245d-404a-9f46-31490b826866 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:06.500863 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded38a73-245d-404a-9f46-31490b826866 service cinder] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:06.511776 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1439/2866] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:43:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:43:07.967408 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe5281-c7c4-485e-8ff6-c65b22a43eff None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:07.968006 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe5281-c7c4-485e-8ff6-c65b22a43eff None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:07.968270 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe5281-c7c4-485e-8ff6-c65b22a43eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:07.995186 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-6bfe5281-c7c4-485e-8ff6-c65b22a43eff None None] MFA Rules not processed for user `3fb39b017d944d7d9808649e85903d43`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:07.996901 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6bfe5281-c7c4-485e-8ff6-c65b22a43eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:08.006593 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1428/2867] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:43:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:08.601033 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded38a73-245d-404a-9f46-31490b826866 service cinder] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:08.602804 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-ded38a73-245d-404a-9f46-31490b826866 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:08.692125 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded38a73-245d-404a-9f46-31490b826866 service cinder] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:08.696947 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0532a75-63d2-462f-8d81-a86840dabc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:08.698121 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0532a75-63d2-462f-8d81-a86840dabc2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:08.698542 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0532a75-63d2-462f-8d81-a86840dabc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:08.699007 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0532a75-63d2-462f-8d81-a86840dabc2c admin admin] PATH_INFO: `/v3/users/cb28e4ff119a431bba4a722c18f5a2a9` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:08.700587 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0532a75-63d2-462f-8d81-a86840dabc2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb28e4ff119a431bba4a722c18f5a2a9)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:08.703266 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0532a75-63d2-462f-8d81-a86840dabc2c admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:08.761625 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d0532a75-63d2-462f-8d81-a86840dabc2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb28e4ff119a431bba4a722c18f5a2a9'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:08.790390 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d0532a75-63d2-462f-8d81-a86840dabc2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb28e4ff119a431bba4a722c18f5a2a9'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:08.791509 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d0532a75-63d2-462f-8d81-a86840dabc2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb28e4ff119a431bba4a722c18f5a2a9'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:08.807796 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d0532a75-63d2-462f-8d81-a86840dabc2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb28e4ff119a431bba4a722c18f5a2a9'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:08.837824 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-d0532a75-63d2-462f-8d81-a86840dabc2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb28e4ff119a431bba4a722c18f5a2a9'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:08.857251 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1440/2868] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:43:08 2022] DELETE /identity/v3/users/cb28e4ff119a431bba4a722c18f5a2a9 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:08.862414 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6fee8-486a-4256-9f82-889402989142 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:08.863013 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6fee8-486a-4256-9f82-889402989142 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:08.863554 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6fee8-486a-4256-9f82-889402989142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:08.894254 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-20c6fee8-486a-4256-9f82-889402989142 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:08.901862 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-20c6fee8-486a-4256-9f82-889402989142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:08.938660 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:08.989104 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1429/2869] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:43:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:09.002003 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0532a75-63d2-462f-8d81-a86840dabc2c admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:09.004196 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-d0532a75-63d2-462f-8d81-a86840dabc2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:09.013698 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:09.090643 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0532a75-63d2-462f-8d81-a86840dabc2c admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:09.096279 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706d757-ffc6-4367-86f6-d064a9cfdecb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:09.097090 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706d757-ffc6-4367-86f6-d064a9cfdecb service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:09.097549 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706d757-ffc6-4367-86f6-d064a9cfdecb service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:09.097955 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706d757-ffc6-4367-86f6-d064a9cfdecb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:09.121663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706d757-ffc6-4367-86f6-d064a9cfdecb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:09.121663 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706d757-ffc6-4367-86f6-d064a9cfdecb service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:09.134944 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1441/2870] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:43:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:43:09.520461 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6fee8-486a-4256-9f82-889402989142 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:09.522262 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-20c6fee8-486a-4256-9f82-889402989142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:09.547286 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6fee8-486a-4256-9f82-889402989142 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:09.563212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5bd912-4fa4-4ebb-b1dc-da5735226078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:09.563212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5bd912-4fa4-4ebb-b1dc-da5735226078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:09.563212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5bd912-4fa4-4ebb-b1dc-da5735226078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:09.563212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5bd912-4fa4-4ebb-b1dc-da5735226078 admin admin] PATH_INFO: `/v3/projects/60629f0ce3364e018f570baf10a29a55` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:09.564477 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5bd912-4fa4-4ebb-b1dc-da5735226078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60629f0ce3364e018f570baf10a29a55)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:09.566091 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5bd912-4fa4-4ebb-b1dc-da5735226078 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:09.619720 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1b5bd912-4fa4-4ebb-b1dc-da5735226078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60629f0ce3364e018f570baf10a29a55', 'request_id': 'req-1b5bd912-4fa4-4ebb-b1dc-da5735226078'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:09.628488 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1b5bd912-4fa4-4ebb-b1dc-da5735226078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60629f0ce3364e018f570baf10a29a55', 'request_id': 'req-1b5bd912-4fa4-4ebb-b1dc-da5735226078'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:09.635870 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1b5bd912-4fa4-4ebb-b1dc-da5735226078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60629f0ce3364e018f570baf10a29a55', 'request_id': 'req-1b5bd912-4fa4-4ebb-b1dc-da5735226078'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:09.647783 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1b5bd912-4fa4-4ebb-b1dc-da5735226078 admin admin] The token cache is being invalidate because project 60629f0ce3364e018f570baf10a29a55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:43:09.648071 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1b5bd912-4fa4-4ebb-b1dc-da5735226078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:09.648806 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-1b5bd912-4fa4-4ebb-b1dc-da5735226078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:09.650550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1430/2871] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:43:09 2022] DELETE /identity/v3/projects/60629f0ce3364e018f570baf10a29a55 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:19.128308 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=24 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:19.644455 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:26.507350 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f348d-a179-4357-8661-9005e82cba79 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:26.507939 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f348d-a179-4357-8661-9005e82cba79 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:26.507939 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f348d-a179-4357-8661-9005e82cba79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:26.526596 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-3d8f348d-a179-4357-8661-9005e82cba79 None None] MFA Rules not processed for user `9de9e460242c4a02ad28af485ad851e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:26.527839 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-3d8f348d-a179-4357-8661-9005e82cba79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:26.597683 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1442/2872] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:43:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:31.346231 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fc779-f09f-4627-af01-757a6e4d8d8a None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:31.346807 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fc779-f09f-4627-af01-757a6e4d8d8a None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:31.347017 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fc779-f09f-4627-af01-757a6e4d8d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:31.367368 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-3c4fc779-f09f-4627-af01-757a6e4d8d8a None None] MFA Rules not processed for user `cfc2f31f654e486d8505d80aad17d17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:31.368612 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-3c4fc779-f09f-4627-af01-757a6e4d8d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:31.431850 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1431/2873] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 17:43:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:31.484857 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f348d-a179-4357-8661-9005e82cba79 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:31.486373 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-3d8f348d-a179-4357-8661-9005e82cba79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:31.563039 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f348d-a179-4357-8661-9005e82cba79 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:31.568420 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5a080-e220-45a1-aa6b-4346b86f56ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a37d1201d54afdad9b228ef4f2d787', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:31.568722 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5a080-e220-45a1-aa6b-4346b86f56ed service swift] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:31.569010 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5a080-e220-45a1-aa6b-4346b86f56ed service swift] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:31.569384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5a080-e220-45a1-aa6b-4346b86f56ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:31.579714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5a080-e220-45a1-aa6b-4346b86f56ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:31.581344 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5a080-e220-45a1-aa6b-4346b86f56ed service swift] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:31.590110 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1443/2874] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 17:43:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:43:32.170596 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12535acf-d4ee-4b8d-b4ba-2d22e78d219d None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:32.171132 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12535acf-d4ee-4b8d-b4ba-2d22e78d219d None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:32.171247 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12535acf-d4ee-4b8d-b4ba-2d22e78d219d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:32.191675 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-12535acf-d4ee-4b8d-b4ba-2d22e78d219d None None] MFA Rules not processed for user `cfc2f31f654e486d8505d80aad17d17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:32.192893 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-12535acf-d4ee-4b8d-b4ba-2d22e78d219d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:32.200534 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1432/2875] 10.222.0.40 () {58 vars in 967 bytes} [Thu Jun 16 17:43:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:32.213622 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5a080-e220-45a1-aa6b-4346b86f56ed service swift] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:32.233416 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5a080-e220-45a1-aa6b-4346b86f56ed service swift] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:32.242231 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a75ed-ac71-4518-82ec-68704f182439 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a37d1201d54afdad9b228ef4f2d787', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:32.243454 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a75ed-ac71-4518-82ec-68704f182439 service swift] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:32.243791 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a75ed-ac71-4518-82ec-68704f182439 service swift] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:32.244297 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a75ed-ac71-4518-82ec-68704f182439 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:32.256645 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a75ed-ac71-4518-82ec-68704f182439 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:32.258301 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a75ed-ac71-4518-82ec-68704f182439 service swift] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:32.268361 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1444/2876] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 17:43:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:43:32.486276 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518a934-5fb1-4ef8-9e55-126502bc6208 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:32.486839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518a934-5fb1-4ef8-9e55-126502bc6208 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:32.486839 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518a934-5fb1-4ef8-9e55-126502bc6208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:32.508119 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-6518a934-5fb1-4ef8-9e55-126502bc6208 None None] MFA Rules not processed for user `cfc2f31f654e486d8505d80aad17d17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:32.509396 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-6518a934-5fb1-4ef8-9e55-126502bc6208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:32.518264 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1433/2877] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 17:43:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:32.531649 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a75ed-ac71-4518-82ec-68704f182439 service swift] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:32.543570 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a75ed-ac71-4518-82ec-68704f182439 service swift] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:32.549092 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75546437-fd37-42ad-8220-524a285b3725 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a37d1201d54afdad9b228ef4f2d787', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:32.549778 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75546437-fd37-42ad-8220-524a285b3725 service swift] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:32.550071 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75546437-fd37-42ad-8220-524a285b3725 service swift] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:32.550355 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75546437-fd37-42ad-8220-524a285b3725 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:32.559919 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75546437-fd37-42ad-8220-524a285b3725 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:32.561609 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75546437-fd37-42ad-8220-524a285b3725 service swift] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:32.570816 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1445/2878] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 17:43:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:43:32.907574 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ce9af-aa17-455e-851b-6031094b8c32 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:32.907951 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ce9af-aa17-455e-851b-6031094b8c32 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:32.908205 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ce9af-aa17-455e-851b-6031094b8c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:32.928051 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-868ce9af-aa17-455e-851b-6031094b8c32 None None] MFA Rules not processed for user `cfc2f31f654e486d8505d80aad17d17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:32.929635 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-868ce9af-aa17-455e-851b-6031094b8c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:32.942151 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1434/2879] 10.222.0.40 () {58 vars in 967 bytes} [Thu Jun 16 17:43:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:43:32.954908 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75546437-fd37-42ad-8220-524a285b3725 service swift] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:32.967996 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75546437-fd37-42ad-8220-524a285b3725 service swift] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:32.973472 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bc549-ad8d-4985-a4cd-044555d5a908 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a37d1201d54afdad9b228ef4f2d787', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:32.974204 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bc549-ad8d-4985-a4cd-044555d5a908 service swift] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:32.974519 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bc549-ad8d-4985-a4cd-044555d5a908 service swift] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:32.974907 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bc549-ad8d-4985-a4cd-044555d5a908 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:32.985731 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bc549-ad8d-4985-a4cd-044555d5a908 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:32.987352 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bc549-ad8d-4985-a4cd-044555d5a908 service swift] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:32.998711 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1446/2880] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 17:43:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:43:35.216182 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ce9af-aa17-455e-851b-6031094b8c32 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:35.218200 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-868ce9af-aa17-455e-851b-6031094b8c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:35.296828 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ce9af-aa17-455e-851b-6031094b8c32 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:35.301407 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4c9c9-3cdd-4794-986a-c2a0932cf8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:35.302302 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4c9c9-3cdd-4794-986a-c2a0932cf8c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:35.302587 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4c9c9-3cdd-4794-986a-c2a0932cf8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:35.302869 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4c9c9-3cdd-4794-986a-c2a0932cf8c0 admin admin] PATH_INFO: `/v3/users/3fb39b017d944d7d9808649e85903d43` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:35.304069 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4c9c9-3cdd-4794-986a-c2a0932cf8c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fb39b017d944d7d9808649e85903d43)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:35.306145 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4c9c9-3cdd-4794-986a-c2a0932cf8c0 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:35.357440 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-faf4c9c9-3cdd-4794-986a-c2a0932cf8c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fb39b017d944d7d9808649e85903d43'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:35.369608 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-faf4c9c9-3cdd-4794-986a-c2a0932cf8c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fb39b017d944d7d9808649e85903d43'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:35.370337 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-faf4c9c9-3cdd-4794-986a-c2a0932cf8c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fb39b017d944d7d9808649e85903d43'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:35.378051 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-faf4c9c9-3cdd-4794-986a-c2a0932cf8c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fb39b017d944d7d9808649e85903d43'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:35.385926 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-faf4c9c9-3cdd-4794-986a-c2a0932cf8c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fb39b017d944d7d9808649e85903d43'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:35.393010 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1435/2881] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:43:35 2022] DELETE /identity/v3/users/3fb39b017d944d7d9808649e85903d43 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:35.398277 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bcb7d5-1f0c-4f69-b2d1-11ccd63122a5 None None] REQUEST_METHOD: `POST` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:35.399242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bcb7d5-1f0c-4f69-b2d1-11ccd63122a5 None None] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:35.399242 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bcb7d5-1f0c-4f69-b2d1-11ccd63122a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:35.417533 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.auth.core [None req-68bcb7d5-1f0c-4f69-b2d1-11ccd63122a5 None None] MFA Rules not processed for user `eac0995e068547e3890aa6066bdecd84`. Rule list: `[]` (Enabled: `True`). {{(pid=80152) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:35.419416 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-68bcb7d5-1f0c-4f69-b2d1-11ccd63122a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:35.472046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=100 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:35.484791 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1447/2882] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 17:43:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:35.494533 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4c9c9-3cdd-4794-986a-c2a0932cf8c0 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:35.496007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-faf4c9c9-3cdd-4794-986a-c2a0932cf8c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:35.503602 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:35.556653 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4c9c9-3cdd-4794-986a-c2a0932cf8c0 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:35.561081 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed934d26-2a46-4cef-aa66-d77fc1487104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:35.562361 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed934d26-2a46-4cef-aa66-d77fc1487104 service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:35.562659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed934d26-2a46-4cef-aa66-d77fc1487104 service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:35.562942 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed934d26-2a46-4cef-aa66-d77fc1487104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:35.572550 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed934d26-2a46-4cef-aa66-d77fc1487104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:35.574031 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed934d26-2a46-4cef-aa66-d77fc1487104 service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:35.584655 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1436/2883] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:43:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:43:35.800148 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bcb7d5-1f0c-4f69-b2d1-11ccd63122a5 None None] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:35.801755 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-68bcb7d5-1f0c-4f69-b2d1-11ccd63122a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:35.815367 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bcb7d5-1f0c-4f69-b2d1-11ccd63122a5 None None] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:35.821482 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8cddf-e222-4e51-a2fd-64384e495c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:35.822499 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8cddf-e222-4e51-a2fd-64384e495c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:35.822964 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8cddf-e222-4e51-a2fd-64384e495c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:35.823654 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8cddf-e222-4e51-a2fd-64384e495c30 admin admin] PATH_INFO: `/v3/projects/2d208bc7521b47589bc3bcf5394fd361` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:35.825938 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8cddf-e222-4e51-a2fd-64384e495c30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d208bc7521b47589bc3bcf5394fd361)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:35.828052 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8cddf-e222-4e51-a2fd-64384e495c30 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:35.895748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-2de8cddf-e222-4e51-a2fd-64384e495c30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d208bc7521b47589bc3bcf5394fd361', 'request_id': 'req-2de8cddf-e222-4e51-a2fd-64384e495c30'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:35.904372 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-2de8cddf-e222-4e51-a2fd-64384e495c30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d208bc7521b47589bc3bcf5394fd361', 'request_id': 'req-2de8cddf-e222-4e51-a2fd-64384e495c30'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:35.914521 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-2de8cddf-e222-4e51-a2fd-64384e495c30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d208bc7521b47589bc3bcf5394fd361', 'request_id': 'req-2de8cddf-e222-4e51-a2fd-64384e495c30'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:35.933453 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-2de8cddf-e222-4e51-a2fd-64384e495c30 admin admin] The token cache is being invalidate because project 2d208bc7521b47589bc3bcf5394fd361 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:43:35.934103 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-2de8cddf-e222-4e51-a2fd-64384e495c30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:35.935601 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-2de8cddf-e222-4e51-a2fd-64384e495c30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:35.938023 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1448/2884] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:43:35 2022] DELETE /identity/v3/projects/2d208bc7521b47589bc3bcf5394fd361 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:45.580240 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=24 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:45.928336 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=29,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:57.810669 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d3e2c-ae0e-467b-b188-c342ddf748a2 None None] REQUEST_METHOD: `POST` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:57.811178 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d3e2c-ae0e-467b-b188-c342ddf748a2 None None] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:57.811286 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d3e2c-ae0e-467b-b188-c342ddf748a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:57.829132 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.auth.core [None req-b72d3e2c-ae0e-467b-b188-c342ddf748a2 None None] MFA Rules not processed for user `cfc2f31f654e486d8505d80aad17d17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80151) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:57.830371 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b72d3e2c-ae0e-467b-b188-c342ddf748a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:57.888564 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1437/2885] 10.222.0.40 () {58 vars in 967 bytes} [Thu Jun 16 17:43:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:57.902406 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8cddf-e222-4e51-a2fd-64384e495c30 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:57.904537 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-2de8cddf-e222-4e51-a2fd-64384e495c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:57.975604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8cddf-e222-4e51-a2fd-64384e495c30 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:57.981353 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c310222-b1c9-4214-9f88-542f6d66ad10 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a37d1201d54afdad9b228ef4f2d787', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:57.982008 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c310222-b1c9-4214-9f88-542f6d66ad10 service swift] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:57.982298 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c310222-b1c9-4214-9f88-542f6d66ad10 service swift] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:57.982581 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c310222-b1c9-4214-9f88-542f6d66ad10 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:57.990903 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c310222-b1c9-4214-9f88-542f6d66ad10 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:57.992592 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c310222-b1c9-4214-9f88-542f6d66ad10 service swift] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:58.004470 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1449/2886] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 17:43:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:44:05.878423 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d3e2c-ae0e-467b-b188-c342ddf748a2 None None] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:05.879912 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-b72d3e2c-ae0e-467b-b188-c342ddf748a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:05.957911 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d3e2c-ae0e-467b-b188-c342ddf748a2 None None] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:05.964313 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3f947-a416-432e-9c2c-7e148a844529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:05.965392 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3f947-a416-432e-9c2c-7e148a844529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:05.965687 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3f947-a416-432e-9c2c-7e148a844529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:05.965971 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3f947-a416-432e-9c2c-7e148a844529 admin admin] PATH_INFO: `/v3/users/af7be60f65da43f982c1ec2e91397051` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:05.967238 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f3f947-a416-432e-9c2c-7e148a844529 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af7be60f65da43f982c1ec2e91397051)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:05.969224 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f3f947-a416-432e-9c2c-7e148a844529 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:06.021701 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-15f3f947-a416-432e-9c2c-7e148a844529 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af7be60f65da43f982c1ec2e91397051'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:06.036281 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-15f3f947-a416-432e-9c2c-7e148a844529 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af7be60f65da43f982c1ec2e91397051'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:06.037021 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-15f3f947-a416-432e-9c2c-7e148a844529 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af7be60f65da43f982c1ec2e91397051'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:06.044558 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-15f3f947-a416-432e-9c2c-7e148a844529 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af7be60f65da43f982c1ec2e91397051'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:06.053048 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-15f3f947-a416-432e-9c2c-7e148a844529 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af7be60f65da43f982c1ec2e91397051'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:06.060086 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1438/2887] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:05 2022] DELETE /identity/v3/users/af7be60f65da43f982c1ec2e91397051 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:06.253095 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c310222-b1c9-4214-9f88-542f6d66ad10 service swift] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:06.265758 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c310222-b1c9-4214-9f88-542f6d66ad10 service swift] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:06.328152 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3d1a6-ebbb-4b12-b520-e4d03f74ba46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:06.329057 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3d1a6-ebbb-4b12-b520-e4d03f74ba46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:06.329371 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3d1a6-ebbb-4b12-b520-e4d03f74ba46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:06.329650 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3d1a6-ebbb-4b12-b520-e4d03f74ba46 admin admin] PATH_INFO: `/v3/projects/ad7c8fc06d764104b1c7107b48d8dddc` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:06.330864 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3d1a6-ebbb-4b12-b520-e4d03f74ba46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad7c8fc06d764104b1c7107b48d8dddc)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:06.332774 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3d1a6-ebbb-4b12-b520-e4d03f74ba46 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:06.381589 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c0e3d1a6-ebbb-4b12-b520-e4d03f74ba46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad7c8fc06d764104b1c7107b48d8dddc', 'request_id': 'req-c0e3d1a6-ebbb-4b12-b520-e4d03f74ba46'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:06.388390 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c0e3d1a6-ebbb-4b12-b520-e4d03f74ba46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad7c8fc06d764104b1c7107b48d8dddc', 'request_id': 'req-c0e3d1a6-ebbb-4b12-b520-e4d03f74ba46'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:06.395384 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c0e3d1a6-ebbb-4b12-b520-e4d03f74ba46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad7c8fc06d764104b1c7107b48d8dddc', 'request_id': 'req-c0e3d1a6-ebbb-4b12-b520-e4d03f74ba46'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:06.415344 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c0e3d1a6-ebbb-4b12-b520-e4d03f74ba46 admin admin] The token cache is being invalidate because project ad7c8fc06d764104b1c7107b48d8dddc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:06.415820 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c0e3d1a6-ebbb-4b12-b520-e4d03f74ba46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:06.417560 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-c0e3d1a6-ebbb-4b12-b520-e4d03f74ba46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:06.420356 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1450/2888] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:06 2022] DELETE /identity/v3/projects/ad7c8fc06d764104b1c7107b48d8dddc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:06.424185 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3f947-a416-432e-9c2c-7e148a844529 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:06.426333 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-15f3f947-a416-432e-9c2c-7e148a844529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:06.452922 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3f947-a416-432e-9c2c-7e148a844529 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:06.478456 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:06.546359 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4e5cd-3ffa-4439-bf25-1c6f229bd698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:06.547505 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4e5cd-3ffa-4439-bf25-1c6f229bd698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:06.548347 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4e5cd-3ffa-4439-bf25-1c6f229bd698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:06.548707 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4e5cd-3ffa-4439-bf25-1c6f229bd698 admin admin] PATH_INFO: `/v3/users/7a3db4c2153340eeaf907c2fe0919e41` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:06.551127 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c4e5cd-3ffa-4439-bf25-1c6f229bd698 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a3db4c2153340eeaf907c2fe0919e41)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:06.555163 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c4e5cd-3ffa-4439-bf25-1c6f229bd698 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:06.624778 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-60c4e5cd-3ffa-4439-bf25-1c6f229bd698 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a3db4c2153340eeaf907c2fe0919e41'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:06.644167 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-60c4e5cd-3ffa-4439-bf25-1c6f229bd698 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a3db4c2153340eeaf907c2fe0919e41'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:06.650539 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-60c4e5cd-3ffa-4439-bf25-1c6f229bd698 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a3db4c2153340eeaf907c2fe0919e41'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:06.661840 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-60c4e5cd-3ffa-4439-bf25-1c6f229bd698 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a3db4c2153340eeaf907c2fe0919e41'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:06.670877 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-60c4e5cd-3ffa-4439-bf25-1c6f229bd698 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a3db4c2153340eeaf907c2fe0919e41'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:06.677549 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1439/2889] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:06 2022] DELETE /identity/v3/users/7a3db4c2153340eeaf907c2fe0919e41 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:06.860392 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3d1a6-ebbb-4b12-b520-e4d03f74ba46 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:06.862011 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-c0e3d1a6-ebbb-4b12-b520-e4d03f74ba46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:06.877975 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3d1a6-ebbb-4b12-b520-e4d03f74ba46 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:06.898274 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:06.947011 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e22a1-cec8-4457-8e34-c162831a2089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:06.948036 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e22a1-cec8-4457-8e34-c162831a2089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:06.948333 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e22a1-cec8-4457-8e34-c162831a2089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:06.948714 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e22a1-cec8-4457-8e34-c162831a2089 admin admin] PATH_INFO: `/v3/projects/0c9770e1404c439abaa86ffb75099000` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:06.949985 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e22a1-cec8-4457-8e34-c162831a2089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c9770e1404c439abaa86ffb75099000)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:06.951901 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e22a1-cec8-4457-8e34-c162831a2089 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:07.010728 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-144e22a1-cec8-4457-8e34-c162831a2089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c9770e1404c439abaa86ffb75099000', 'request_id': 'req-144e22a1-cec8-4457-8e34-c162831a2089'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:07.017380 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-144e22a1-cec8-4457-8e34-c162831a2089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c9770e1404c439abaa86ffb75099000', 'request_id': 'req-144e22a1-cec8-4457-8e34-c162831a2089'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:07.026516 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-144e22a1-cec8-4457-8e34-c162831a2089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c9770e1404c439abaa86ffb75099000', 'request_id': 'req-144e22a1-cec8-4457-8e34-c162831a2089'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:07.042997 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-144e22a1-cec8-4457-8e34-c162831a2089 admin admin] The token cache is being invalidate because project 0c9770e1404c439abaa86ffb75099000 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:07.043225 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-144e22a1-cec8-4457-8e34-c162831a2089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:07.043879 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-144e22a1-cec8-4457-8e34-c162831a2089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:07.045499 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1451/2890] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:06 2022] DELETE /identity/v3/projects/0c9770e1404c439abaa86ffb75099000 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:44:16.673697 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:17.038792 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:26.812296 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4e5cd-3ffa-4439-bf25-1c6f229bd698 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:26.813520 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-60c4e5cd-3ffa-4439-bf25-1c6f229bd698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:26.886668 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4e5cd-3ffa-4439-bf25-1c6f229bd698 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:26.953016 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384854d-b21f-459a-84de-4eec0589fe1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:26.953731 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384854d-b21f-459a-84de-4eec0589fe1c service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:26.954018 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384854d-b21f-459a-84de-4eec0589fe1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:26.954297 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384854d-b21f-459a-84de-4eec0589fe1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:26.956442 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7384854d-b21f-459a-84de-4eec0589fe1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:27.054733 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384854d-b21f-459a-84de-4eec0589fe1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:27.056822 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384854d-b21f-459a-84de-4eec0589fe1c service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:27.068837 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1440/2891] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:28.258207 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e22a1-cec8-4457-8e34-c162831a2089 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:28.260225 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-144e22a1-cec8-4457-8e34-c162831a2089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:28.273121 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e22a1-cec8-4457-8e34-c162831a2089 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:28.278351 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fe100-d57f-438a-b27e-2df010679a83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:28.279033 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fe100-d57f-438a-b27e-2df010679a83 service neutron] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:28.279325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fe100-d57f-438a-b27e-2df010679a83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:28.279611 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fe100-d57f-438a-b27e-2df010679a83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:28.281496 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-1d0fe100-d57f-438a-b27e-2df010679a83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:28.296914 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0fe100-d57f-438a-b27e-2df010679a83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:28.298418 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0fe100-d57f-438a-b27e-2df010679a83 service neutron] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:28.310648 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1452/2892] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:29.336659 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384854d-b21f-459a-84de-4eec0589fe1c service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:29.340706 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-7384854d-b21f-459a-84de-4eec0589fe1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:29.423697 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=100 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:29.428961 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384854d-b21f-459a-84de-4eec0589fe1c service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:29.434007 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796c61f-732d-4bd2-ae0f-9aa825c93a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:29.434928 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796c61f-732d-4bd2-ae0f-9aa825c93a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:29.435209 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796c61f-732d-4bd2-ae0f-9aa825c93a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:29.435486 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796c61f-732d-4bd2-ae0f-9aa825c93a54 admin admin] PATH_INFO: `/v3/users/9de9e460242c4a02ad28af485ad851e5` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:29.436683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796c61f-732d-4bd2-ae0f-9aa825c93a54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9de9e460242c4a02ad28af485ad851e5)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:29.438623 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796c61f-732d-4bd2-ae0f-9aa825c93a54 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:29.502338 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e796c61f-732d-4bd2-ae0f-9aa825c93a54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9de9e460242c4a02ad28af485ad851e5'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:29.518043 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e796c61f-732d-4bd2-ae0f-9aa825c93a54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9de9e460242c4a02ad28af485ad851e5'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:29.518967 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e796c61f-732d-4bd2-ae0f-9aa825c93a54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9de9e460242c4a02ad28af485ad851e5'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:29.529812 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e796c61f-732d-4bd2-ae0f-9aa825c93a54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9de9e460242c4a02ad28af485ad851e5'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:29.544162 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-e796c61f-732d-4bd2-ae0f-9aa825c93a54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9de9e460242c4a02ad28af485ad851e5'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:29.553685 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1441/2893] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:29 2022] DELETE /identity/v3/users/9de9e460242c4a02ad28af485ad851e5 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:29.753532 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fe100-d57f-438a-b27e-2df010679a83 service neutron] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:29.766693 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fe100-d57f-438a-b27e-2df010679a83 service neutron] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:29.860962 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93abb1a-edd3-4c6e-b123-2fc8c0afb32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:29.861882 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93abb1a-edd3-4c6e-b123-2fc8c0afb32b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:29.862183 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93abb1a-edd3-4c6e-b123-2fc8c0afb32b admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:29.862466 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93abb1a-edd3-4c6e-b123-2fc8c0afb32b admin admin] PATH_INFO: `/v3/projects/554c281060a446ed9b2d02bad7b8ec1d` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:29.863776 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93abb1a-edd3-4c6e-b123-2fc8c0afb32b admin admin] RBAC: Authorizing `identity:delete_project(project_id=554c281060a446ed9b2d02bad7b8ec1d)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:29.866187 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93abb1a-edd3-4c6e-b123-2fc8c0afb32b admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:29.924203 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a93abb1a-edd3-4c6e-b123-2fc8c0afb32b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '554c281060a446ed9b2d02bad7b8ec1d', 'request_id': 'req-a93abb1a-edd3-4c6e-b123-2fc8c0afb32b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:29.929884 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a93abb1a-edd3-4c6e-b123-2fc8c0afb32b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '554c281060a446ed9b2d02bad7b8ec1d', 'request_id': 'req-a93abb1a-edd3-4c6e-b123-2fc8c0afb32b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:29.936836 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a93abb1a-edd3-4c6e-b123-2fc8c0afb32b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '554c281060a446ed9b2d02bad7b8ec1d', 'request_id': 'req-a93abb1a-edd3-4c6e-b123-2fc8c0afb32b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:29.952123 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a93abb1a-edd3-4c6e-b123-2fc8c0afb32b admin admin] The token cache is being invalidate because project 554c281060a446ed9b2d02bad7b8ec1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80152) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:29.952427 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a93abb1a-edd3-4c6e-b123-2fc8c0afb32b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:29.953115 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-a93abb1a-edd3-4c6e-b123-2fc8c0afb32b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:29.954717 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1453/2894] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:29 2022] DELETE /identity/v3/projects/554c281060a446ed9b2d02bad7b8ec1d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:38.351535 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796c61f-732d-4bd2-ae0f-9aa825c93a54 admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:38.352985 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-e796c61f-732d-4bd2-ae0f-9aa825c93a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:38.417199 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796c61f-732d-4bd2-ae0f-9aa825c93a54 admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:38.469424 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea86e4-4d6b-4601-a145-53f36cd45b5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a303489493ce4cd2b62e876fbc8a7404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:38.470098 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea86e4-4d6b-4601-a145-53f36cd45b5e service neutron] REQUEST_METHOD: `GET` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:38.470386 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea86e4-4d6b-4601-a145-53f36cd45b5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:38.470669 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea86e4-4d6b-4601-a145-53f36cd45b5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:38.472509 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-26ea86e4-4d6b-4601-a145-53f36cd45b5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:38.525652 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:38.554941 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea86e4-4d6b-4601-a145-53f36cd45b5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:38.555226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea86e4-4d6b-4601-a145-53f36cd45b5e service neutron] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:38.565737 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1442/2895] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:39.948876 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:45.060980 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93abb1a-edd3-4c6e-b123-2fc8c0afb32b admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:45.063183 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-a93abb1a-edd3-4c6e-b123-2fc8c0afb32b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:45.079972 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93abb1a-edd3-4c6e-b123-2fc8c0afb32b admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:45.084579 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc25d2d-cc3b-4b1e-bb2b-b560c754be42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:45.085510 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc25d2d-cc3b-4b1e-bb2b-b560c754be42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:45.085814 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc25d2d-cc3b-4b1e-bb2b-b560c754be42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:45.086092 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc25d2d-cc3b-4b1e-bb2b-b560c754be42 admin admin] PATH_INFO: `/v3/users/faf0a0102da24cedaedd0c67273e275b` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:45.087283 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc25d2d-cc3b-4b1e-bb2b-b560c754be42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=faf0a0102da24cedaedd0c67273e275b)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:45.089299 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc25d2d-cc3b-4b1e-bb2b-b560c754be42 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:45.142660 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-fbc25d2d-cc3b-4b1e-bb2b-b560c754be42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faf0a0102da24cedaedd0c67273e275b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:45.152107 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-fbc25d2d-cc3b-4b1e-bb2b-b560c754be42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faf0a0102da24cedaedd0c67273e275b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:45.152723 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-fbc25d2d-cc3b-4b1e-bb2b-b560c754be42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faf0a0102da24cedaedd0c67273e275b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:45.160092 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-fbc25d2d-cc3b-4b1e-bb2b-b560c754be42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faf0a0102da24cedaedd0c67273e275b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:45.172369 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-fbc25d2d-cc3b-4b1e-bb2b-b560c754be42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faf0a0102da24cedaedd0c67273e275b'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:45.178346 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1454/2896] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:45 2022] DELETE /identity/v3/users/faf0a0102da24cedaedd0c67273e275b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:45.367756 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea86e4-4d6b-4601-a145-53f36cd45b5e service neutron] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:45.381566 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea86e4-4d6b-4601-a145-53f36cd45b5e service neutron] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:45.449415 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620088cb-4791-4030-a4d1-66a2547d1aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:45.450212 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620088cb-4791-4030-a4d1-66a2547d1aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:45.450495 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620088cb-4791-4030-a4d1-66a2547d1aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:45.450776 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620088cb-4791-4030-a4d1-66a2547d1aee admin admin] PATH_INFO: `/v3/projects/314e9434e0ec4c1f9e8d6037143f5ebb` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:45.452027 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620088cb-4791-4030-a4d1-66a2547d1aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=314e9434e0ec4c1f9e8d6037143f5ebb)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:45.453993 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620088cb-4791-4030-a4d1-66a2547d1aee admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:45.505362 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-620088cb-4791-4030-a4d1-66a2547d1aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '314e9434e0ec4c1f9e8d6037143f5ebb', 'request_id': 'req-620088cb-4791-4030-a4d1-66a2547d1aee'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:45.514273 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-620088cb-4791-4030-a4d1-66a2547d1aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '314e9434e0ec4c1f9e8d6037143f5ebb', 'request_id': 'req-620088cb-4791-4030-a4d1-66a2547d1aee'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:45.521327 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-620088cb-4791-4030-a4d1-66a2547d1aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '314e9434e0ec4c1f9e8d6037143f5ebb', 'request_id': 'req-620088cb-4791-4030-a4d1-66a2547d1aee'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:45.532559 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-620088cb-4791-4030-a4d1-66a2547d1aee admin admin] The token cache is being invalidate because project 314e9434e0ec4c1f9e8d6037143f5ebb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:45.532853 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-620088cb-4791-4030-a4d1-66a2547d1aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:45.533543 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-620088cb-4791-4030-a4d1-66a2547d1aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:45.535116 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1443/2897] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:45 2022] DELETE /identity/v3/projects/314e9434e0ec4c1f9e8d6037143f5ebb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:45.539164 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc25d2d-cc3b-4b1e-bb2b-b560c754be42 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:45.540604 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-fbc25d2d-cc3b-4b1e-bb2b-b560c754be42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:45.554574 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc25d2d-cc3b-4b1e-bb2b-b560c754be42 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:45.617284 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6fb48e-7afa-4f50-baa8-40cac744d003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:45.618181 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fb48e-7afa-4f50-baa8-40cac744d003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:45.618467 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fb48e-7afa-4f50-baa8-40cac744d003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:45.618750 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fb48e-7afa-4f50-baa8-40cac744d003 admin admin] PATH_INFO: `/v3/users/f0eec822d5ec487782e6b0015dc5f421` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:45.620224 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6fb48e-7afa-4f50-baa8-40cac744d003 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0eec822d5ec487782e6b0015dc5f421)` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:45.622208 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6fb48e-7afa-4f50-baa8-40cac744d003 admin admin] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:45.677559 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-4a6fb48e-7afa-4f50-baa8-40cac744d003 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0eec822d5ec487782e6b0015dc5f421'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:45.685748 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-4a6fb48e-7afa-4f50-baa8-40cac744d003 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0eec822d5ec487782e6b0015dc5f421'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:45.686379 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-4a6fb48e-7afa-4f50-baa8-40cac744d003 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0eec822d5ec487782e6b0015dc5f421'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:45.693938 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-4a6fb48e-7afa-4f50-baa8-40cac744d003 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0eec822d5ec487782e6b0015dc5f421'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:45.702529 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:45.707922 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.notifications [None req-4a6fb48e-7afa-4f50-baa8-40cac744d003 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0eec822d5ec487782e6b0015dc5f421'} {{(pid=80152) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:45.715843 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1455/2898] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:45 2022] DELETE /identity/v3/users/f0eec822d5ec487782e6b0015dc5f421 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:45.909339 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620088cb-4791-4030-a4d1-66a2547d1aee admin admin] Authenticating user token {{(pid=80151) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:45.910956 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.fernet_utils [None req-620088cb-4791-4030-a4d1-66a2547d1aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80151) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:45.931897 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620088cb-4791-4030-a4d1-66a2547d1aee admin admin] Validating token access rules against request {{(pid=80151) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:45.998120 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6454b-04d0-48ae-95c1-a46702f8ff18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac0995e068547e3890aa6066bdecd84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ada52fc9b44bcbbe7f2cc5f37ab303', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80151) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:45.999123 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6454b-04d0-48ae-95c1-a46702f8ff18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:45.999409 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6454b-04d0-48ae-95c1-a46702f8ff18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:45.999683 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6454b-04d0-48ae-95c1-a46702f8ff18 admin admin] PATH_INFO: `/v3/projects/bbbc2713d5854dd49b08622f859e1ce1` {{(pid=80151) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:46.000921 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e6454b-04d0-48ae-95c1-a46702f8ff18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbbc2713d5854dd49b08622f859e1ce1)` {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:46.002985 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e6454b-04d0-48ae-95c1-a46702f8ff18 admin admin] RBAC: Authorization granted {{(pid=80151) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:46.033169 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:46.069176 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-05e6454b-04d0-48ae-95c1-a46702f8ff18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbbc2713d5854dd49b08622f859e1ce1', 'request_id': 'req-05e6454b-04d0-48ae-95c1-a46702f8ff18'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:46.078584 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-05e6454b-04d0-48ae-95c1-a46702f8ff18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbbc2713d5854dd49b08622f859e1ce1', 'request_id': 'req-05e6454b-04d0-48ae-95c1-a46702f8ff18'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:46.085599 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-05e6454b-04d0-48ae-95c1-a46702f8ff18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbbc2713d5854dd49b08622f859e1ce1', 'request_id': 'req-05e6454b-04d0-48ae-95c1-a46702f8ff18'} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:46.097125 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-05e6454b-04d0-48ae-95c1-a46702f8ff18 admin admin] The token cache is being invalidate because project bbbc2713d5854dd49b08622f859e1ce1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80151) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:46.097454 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-05e6454b-04d0-48ae-95c1-a46702f8ff18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:46.098144 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG keystone.notifications [None req-05e6454b-04d0-48ae-95c1-a46702f8ff18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80151) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:46.100226 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: [pid: 80151|app: 0|req: 1444/2899] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:45 2022] DELETE /identity/v3/projects/bbbc2713d5854dd49b08622f859e1ce1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:55.715014 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:56.093918 dv-cinder-837546-2-iscsi devstack@keystone.service[80151]: DEBUG dbcounter [-] [80151] Writing DB stats keystone:SELECT=18,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80151) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:23.380650 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6fb48e-7afa-4f50-baa8-40cac744d003 admin admin] Authenticating user token {{(pid=80152) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:23.382314 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-4a6fb48e-7afa-4f50-baa8-40cac744d003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:23.540312 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6fb48e-7afa-4f50-baa8-40cac744d003 admin admin] Validating token access rules against request {{(pid=80152) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:23.604046 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad8413-1742-48c7-9b87-ab9dfac6c237 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75f0586a8c1941448d750918183f6dc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a535a305e740368b7a4d77d0ad22a0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80152) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:23.604735 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad8413-1742-48c7-9b87-ab9dfac6c237 service placement] REQUEST_METHOD: `GET` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:23.605320 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad8413-1742-48c7-9b87-ab9dfac6c237 service placement] SCRIPT_NAME: `/identity` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:23.605608 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad8413-1742-48c7-9b87-ab9dfac6c237 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80152) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:23.607448 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.fernet_utils [None req-5dad8413-1742-48c7-9b87-ab9dfac6c237 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80152) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:23.620682 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad8413-1742-48c7-9b87-ab9dfac6c237 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:23.622337 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad8413-1742-48c7-9b87-ab9dfac6c237 service placement] RBAC: Authorization granted {{(pid=80152) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:23.633695 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: [pid: 80152|app: 0|req: 1456/2900] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 17:47:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:47:33.628325 dv-cinder-837546-2-iscsi devstack@keystone.service[80152]: DEBUG dbcounter [-] [80152] Writing DB stats keystone:SELECT=89 {{(pid=80152) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}